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I. INTRODUCTION 


A. BACKGROUND 

As seen in several of the recent wars and conflicts, 
speed and timing are crucial in modern warfare. In the 
Falklands War, the lack of time available to react to a 
threat caused the loss of HMS SHEFFIELD. The HMS SHEFFIELD 
was sunk by fires that could not be brought under control as 
a result of a strike by an Exocet missile. Even though the 
ship had weapons systems that could have defeated the 
Exocet, its inability to initially detect the missile at a 
far range rendered these defenses useless. The Electronic 
Warfare (EW) operators on the SHEFFIELD had little warning 
of the Exocet due to self-induced jamming. When the self- 
jamming (inadvertent, of course) ceased, the Exocet was 
immediately detected, but it was too late to engage. The 
missile struck about ten seconds later. Although 
technologically superior, the British did not correctly 
manage the Radio Frequency (RF) spectrum and lost a ship. 
The self-jamming was caused by equipment interference and 
was either not noticed earlier or dismissed as unlikely to 
cause serious problems. This problem and others like it are 
now being resolved by the British. 

In war at sea today, it is necessary to provide an 


adequate reaction time. Reaction time is the time between 
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detection of the incoming target and weapons engagement. 
Response time is the time between enemy weapons release and 
impact (i.e., the time available for detection, reaction, 
and engagement). In the example cited above, that adequate 
reaction time simply was not present. In the case of HMS 
SHEFFIELD, improper management of the electromagnetic (EM) 
spectrum set up the situation of inadequate reaction time. 
With the coming of sophisticated weapon systems and 
supersonic missiles, the amount of time available to respond 
to a threat has been steadily reduced. In World War II 
reaction time could be measured on the order of dozens of 
minutes. Today reaction times are on the order of dozens of 
seconds. With initial detection at the horizon,  sea- 
skimming missiles offer only 30 seconds warning before 
impact. Today’s Combat Information Center (CIC) needs to be 
organized in such a way as to derive maximum efficiency and 
speed from operations in order to reduce reaction time as 
much as possible. 

The problem of reduced reaction time is not new and 
equipments in many areas has been developed to meet this 
need. The Navy Tactical Data System (NTDS) was developed to 
solve this problem. It reduces the amount of time needed to 
understand the tactical environment around the ship and by 
providing a more complete picture, aids sound decision 


making. 
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With the development and installation of effective 
defensive weapon systems onboard Navy ships, effort must now 


be devoted to the reduction of reaction time. Effective 


weapons are available, only the time to employ them 
correctly is needed. Effective long range sensor systems 
can provide adequate warning and “buy” time for the 
employment of the appropriate weapon. Therefore, it can be 


concluded that anything that "buys time" is of value. 

But how does one buy time? There are two ways: (1) 
machines can be built to react more quickly or, €2) 
operators can be trained to respond faster. Although 
syatens will help the Fleet sailor react quicker, there is a 
limit as to how fast he can respond. Working spaces need to 
be optimized so that the sailor can respond optimally. In 
this context, efficiency translates into speed which 
translates into reduced reaction tine. The efficient 
arrangement of equipment in a working space has not been 
addressed by the Navy in such a way as to promote effective 
and efficient spaces. (For the remainder of this effort, 
the term "working space” or "work space" will be used to 
denote a combat space where data is searched for, collected, 
evaluated, disseminated, and/or acted upon.) An efficient 
and effectively laid out workspace will, intuitively, buy 
time. The barriers imposed by improper design and poor 


layout can never be totally compensated for by training. 
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B. NEED 

During the procurement cycle, there is a requirement to 
perform human factors engineering on all new equipment to 
insure an adequate man-machine interface. However, there is 
inadequate methodology for insuring that space arrangement 
contributes to successful mission accomplishment. Space 
arrangement is, apparently, dictated primarily by the need 
to fit new and existing equipment into a space. This is not 
intended to belittle the efforts of those who are charged 
with designing the layouts and arrangements of combat 
spaces, but is intended to address methods for improving the 
efficiencies of layouts. 

Before the layouts of combat spaces can be redesigned, 
it is first necessary to determine if there is a deficiency 
in the existing layout. A measure of adequacy of space 
arrangements must be developed. At present there is no such 
measure for combat systems layouts. 

The field of human factors engineering has developed 
techniques for assessing the adequacy of tasks, subsystems, 
systems, and organizations. However, due to the dynamic 


nature of shipboard work space development there appears to 


have been few human factors engineering techniques 
addressing the arrangement of systems as applied to the 
space as a whole or the entire mission work areas. This is 


due to the fact that new equipment with new functions and 
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increased capabilities are constantly being introduced into 
spaces barely adequate for the original equipment. 

As a result of the procurement cycle, human factors 
engineering is applied only on single systems or consoles. 
It has been recognized that training personnel to overcome 
the human factors design deficiencies is not cost effective 
in terms of either time, money, or manpower. The current 
requirement is for total individual system analysis in the 
areas of compatibility, interoperability, and human factors. 
It is mainly in the area of integrating these systems 
together in a work space that significant improvement is 


needed. 


C. PURPOSE OF THESIS 

The purpose of this thesis is to present and discuss a 
method of assessing the effectiveness of a work space 
layout. In addition, this method will provide the framework 
for pinpointing those areas where redesign will be most cost 


effective. 
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II. THE NATURE OF THE DIFFICULTY 


A. CURRENT METHOD 

How are space layouts currently assessed in the United 
States Navy? Or, perhaps, a better question is: is the 
current method of assessing space arrangements adequate? 
What is the current method? 

There are two methods of improving a space layout. The 
first method is by fleet inputs. There are no formal 
procedures as such. To initiate a design change,a request 
for change (no specific format) is submitted by the 
individual (or ship) via his chain of command to the 
appropriate engineering office within the Naval Sea Systems 
Command (NAVSEASYSCOM). An engineer studies the proposal to 
see if it has merit. If it does, he forwards it within 
NAVSEASYSCOM and through the appropriate system command. 
From this point, if it is acceptable, it is forwarded to the 
Office of the Chief of Naval Operations (OPNAV). OPNAV is 
the configuration manager for all platforms and makes the 
final decision on configuration or layout design changes. 
For example, during the Board of Inspection and Survey (BIS) 
Acceptance Trials for USS CARL VINSON (CVN-70), a strong 
case for redesigning the CIC arrangement was made by the 
commissioning crew. In particular, the cramped space of the 


EW Module on CARL VINSON was addressed and a solution 
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proposed. That solution apparently was forwarded to 
NAVSEASYSCOM and Space Naval Warfare Command and then to 
OPNAV because it has been incorporated in the layout of the 
EW Module on CVN-71 (See Figure 1). 

The other method in improving space design is a mock-up 
approach. At various organizations, mock-ups are used to 
test the layout designs considered. These organizations are 
under contract to produce a specific kind of mock-up. The 
Naval Ocean Systems Center (NOSC) in San Diego does some 
mock-up work under the direction of  NAVSEASYSCOM. At 
present they have a carrier Combat Information Center (CIC) 
mock-up. It contains the Display and Decision portion which 
includes the Surface Warfare Module and the Air Warfare 
Module. It does not contain Detection and Tracking, the 
Electronic Warfare Module, or the Anti-Submarine Warfare 


Module. 


B. DEFICIENCIES 

The current method of improving the layout/arrangement 
design of some of the spaces on our surface ships, has four 
major shortcomings: (1) lack of user input, (2) lack of 
human factors engineering, (3) lack of a learning curve, and 
(4) no planning for growth. These deficiencies and the 
impact they have on the effectiveness of the space is now 


discussed. 
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Figure 1. EW Module Layout (Proposed) 
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lve ack of User Input 


The current method of layout improvement has little 
or no operator/user input. There is, perhaps, some user 
input as in the CARL VINSON CIC example cited above but this 
appears to be the exception rather than the rule. There is 
no formal method of submitting a design change through 
normal Navy channels to NAVSEA. This is a very serious 
deficiency because sketchy or total lack of fleet input is 
counter-productive. The design of a space by those who do 
not and will not be using it has a tendency to result in a 
far from optimal design. For example, a radio or “bitch 
box” that is frequently used is placed just out of reach. 
In the EW Module on USS CARL VINSON, the 12 MC (internal 
communications set) used to communicate with the Tactical 
Action Officer (TAQ) and INTELCamong others) is placed such 
that the EW Supervisor and NTDS operator have to get up out 
of his seat to talk. There is, in fairness, a hand mike 
that can be attached but this has the undesirable side 
effect of cluttering the workstation. 


22 Lack of Human Factors Engineering 


The only human factors engineering being employed is 


basic. This method has been characterized as "moving the 
furniture around". This is done until there is an 
apparently  workable solution. Again there is inadequate 


fleet or operator inputs to check the "new" arrangement. It 


must be said that those employing this method are trying to 


18 


find an arrangement that facilitates an efficient and 
effective operation. This method is the best method 
currently available to accomplish this task, but still 
something is lacking. 
3. Lack of a Learning Curve 

However good the results of the mock-up method may 
be, there is no apparent learning curve in successive 
layouts. For example, the EW aia yout on USS AMERICA 
provided the necessary room for the activities of EW and 
gave the impression of smooth efficiency and competence. 
However, on later aircraft carriers (most notably, the 
NIMITZ class) the EW Module arrangement is a regression and 
in nowise approaches the room and layout effectiveness found 
on AMERICA. If the NIMITZ class carrier EW Module layout 
was intended to be an improvement over AMERICA, it failed. 

4. No plan for Growth 

The current method is deficient in terms of its 
potential for growth. Few designs provide room for 
expansion for either new equipment or modifications to older 
equipment. When new equipment is added, the space for it 
must come from someplace, even if that area has another 
function. Simply adding new equipment does not aid the 
operation of the overall work area and may even be 
counterproductive in that efficiency may be reduced. 

When lead units are designed and built, they are 


constructed with only existing equipment is mind (this is a 
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general rule and there are some exceptions). There is some 
small amount of room for expansion of capabilities but it is 
thought that the new equipment will replace older equipment 
and take up the same amount (or less) of room and fit into 
the same space. This thought does not take into account new 
missions for the space with corresponding new equipment, new 
capabilities, and new space requirements. Therefore, one 
can readily see that new aduiP ment must be added wherever 
there is room for it. Sometimes the space where the new 
equipment is added is unsatisfactory for the equipment and 
its operation. By way of example, USS CARL VINSON is a 
NIMITZ class aircraft carrier and was built using 
essentially the same blueprints as the lead ship. The EW 
Module space waa not changed even though the SLQ-17 and SSQ- 
82 (MUTE), not yet procured when NIMITZ was designed and 
built, were slated for CARL VINSON (and all carriers, 
eventually) (See Figure 2). Only one equipment rack was 
removed to make way for both equipments. The SLQ-17 console 
fits into the vacated equipment rack. This still left the 
computer unit (about one and a half racks) and MUTE (which 
was designed and built wider than the standard rack) to be 
placed somewhere within the EW Module. The SLQ-17 computer 
rack was placed against the bulkhead in the middle of the 
space. At most, three operators could fit comfortably into 
the EW Module even though General Quarters manning calls for 


four operators, and MUTE was placed outside the EW Module in 
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a passageway within CIC that was heavily traveled. There 
was simply no room in the EW Module for both the EW 
operators and MUTE--one or the other had to go. 

In addition to the problems cited above, to 
accommodate the inclusion of all the equipment in the EW 
Module, some severe space economies had to be made. The 
layout now took on the appearance as shown in Figure 2. To 
allow some passage of operators and maintenance people among 
and around the equipment, a "straight line" layout was 
adopted. This had the sole advantage of allowing all the 
equipment possible to be place in the space. However, the 
question can logically be asked, “Does auch an arrangement 
add or detract from the efficiency and effectiveness of 
space utilization in accomplishing the mission?” New 
equipment added to a space that was not designed for it may 
cause integration problems due to its intrinsic nature 
(i.e., in the equipment itself), its new location (e.g., the 
SLQ-17 computer rack), and reduced workspace (in our 
example, several racks where one used to be to the exclusion 
of another piece of equipment--MUTE). 

The remainder of this thesis will be given over to 
attempting to find a workable solution to the problem of 
adequately designing a work space, in particular, an EW 
Module. As indicated earlier this is an area where the costs 
are in dollars and effort, but the payoff is in shorter 


reaction time and, ultimately, in ships and lives saved. 
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III. APPROACH 


A. IMPROVEMENT TO LAYOUTS 

The solution to layout/arrangement improvement is 
neither simple nor straightforward. An improvement, 
however, can be found in a threefold approach to the 
problem. These are: (1) a ship class, module, mock-up at a 
land based laboratory, (2) fleet inputs added to it on a 
regular basis, and (3) a quantifiable measure that can be 
used to determine overall effectiveness and pinpoint problem 
areas. 

Establishing a class, module, mock-up at a land based 
laboratory makes good sense. Here, the results of several 
mock-ups can be stored and compared. Here, too, a "learning 
curve” can be established. What does not work for one class 
and module may never work, or it may work for another class 
Ship and another module. The cost of mock-ups can be kept 
low. Mock-ups of new equipment entering the fleet can be 
sent to just one location and then incorporated into the 
design or redesign. Mock-ups of new ship classes can easily 
be done there. 

NOSC at San Diego seems to be a good place to have this 
mock-up facility for several reasons. First, experts there 
have already done some mock-up work and have a certain 


amount of experience in this area. Secondly, they are near 
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a good source of fleet inputs in San Diego. Once a mock-up 
was designed (or E NOSC could request some fleet 
operatora from one of the ships of that particular class and 
these operators could critique the mock-up and make 
suggestions for improvement. For added realism and 
additional inputs, a mock scenario could be played out by 
the operators on the mock-up. This has the added possible 
benefit of uncovering any oversight by either NOSC or the 
operators’ critique. The two logical places for the mock-up 
site are Norfolk, Va. and San Diego, Ca. 

Fleet inputs in the design/redeaign of the layout 
process is of the utmost importance. The fleet operators 
are the people who have to use the equipment and accomplish 
the mission within the space. They, from the benefit of 
several years individual and many years collective 
experience, will be able to note problems with the mock-up 
that the deaignera may have miaaed. Designera of single 
equipments tend to think of their equipment in isolation 
from all othera. Layout deaignera are often not familiar 
with the operating characteristics of all the equipment. 
Fleet operators suffer from neither of theae deficiencies. 
However, operatora do have a bias toward doing things as 
they are currently done and may resist change. Nevertheless, 
they are still probably the best ones to evaluate the mock- 


up. 
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As seemingly complete as the combination of both 
laboratory mock-up and fleet input might be, there is one 
more area that needs to be covered. This is a quantitative 
assessment of the present layout and mock-up layouts. There 
are several reasons for this. First, a quantitative 
assessment of a present layout may indicate that it does not 
need improving or that the cost of improving the layout is 
not justified by the amount of improvement. Second, a 
quantitative assessment based in part on questionnaires to 
fleet operators may awaken thoughts of some inadequacy that 
was not present in the conscious memory but was tucked away 
in the recesses of the mind. Finally, a quantitative 
assessment is necessary to be able to compare functional 
layouts one to another. The final aspect of this approach is 
a way of assessing the effectiveness of the layout. 

Various techniques have been developed that will aid in 
assessing effectiveness. However, these methods have been 
used on systems that are dissimilar to those found on ships 
and must be modified. The method that will be utilized is a 
combination of three different but related techniques: Task 
Analysis, Link Analysis, and Operability Analysis. Two 
major studies have been reviewed to determine the extent of 
these analyses and how they might be modified for a layout 
improvement application. These are Integration Analysis and 


Mission  Operability Assessment Technique (MOAT). A brief 
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look at each of these will indicate the salient portions of 
each for this application. 
Integration Analysis is the integration of Task 
Analysis, Operator Interviews, and Link Analysis to evaluate 
a system’s Functional Mock-up. Integration Analysis was 
designed as a viable Test and Evaluation technique for the 
earlier stages of Developmental Test and Evaluation (DT&E) 
in order to reduce design discrepancies and minimize 
acquistion costs and time (Ref. 1]. 
MOAT, an evaluation methodology, measures the 
operability of a system or subsystem in terms of operator 
tasks performed during a mission. It essentially is an 
Operability Analysis. 
In general, MOAT addresses the problem of how well a 
operator can use a system or subsystem to perform task 
within the mission context. Contrasted to evaluation 
using human engineering design criteria which present onl 
pass or fail information, this technique provide 
information on the degree of system and/or  subsyste 
success or failure. (Ref. 2i:pp. 3-4] 

The underlying techniques of task analysis, scaling 

methodology, and multi-attribute utility (MAU) theory have 

been integrated into one comprehensive methodology. 
MOAT systematically structures operator tasks in accordance 
with mission needs and then assesses the operability o 
these tasks through conjoint measurement. All assessment 
are then integrated within rules established through MA 


theory. The output of MOAT is quantitative informatio 
about the operability of an entire aystem, auch as 


fighter or attack aircraft; the operability of specific 


p 


subsystems auch as radar, communications, or navigation 


and; finally, the operability of each task performed during 


r 


a mission phase. In short, MOAT measures subayatem and/o 
system goal attainment. (Ref. 2ip. 4] 
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Having briefly described both Integration Analysis and 
MOAT, parts of each were combined in this thesis to 
provide a technique that is well suited to a layout 
improvement application. From MOAT the techniques to assess 
man-system compatibility (i.e., Task Analysis and 
Operability Analysis?) were used. This waa considered in a 
larger context in order to assess man as a team rather than 
as an individual. Operability Analysis consists of two 
parts: multi-attribute utility (MAU) theory ‘to be discussed 
later), and scaling theory. The use of questionnaires and 
Link Analysis came from nación Analysis. Note that 
both Integration Analysis and MOAT contain Task Analysis. 
The questionnaires serve two purposes. First, they focus 
attention on the problem areas of the design/arrangement. 
Secondly, the completed questionnaires support assessment of 
the layout effectiveness. Finally, the questionnaires form 
a link between the various analyses and operator inputs. 

MOAT was designed to assess man-aystem compatibility. 
The original MOAT used a construct that embodied the three 
most important divisions of the man-system  compatibility-- 
man, system, and mission and how all three interact during 
mission accomplishment. The difference considered here lies 
in the evaluation of a team of operators rather than just 
one man and the fact that a group of subsystems arranged in 
a particular manner is used to determine the  operability. 


Hence, the three most important divisions of the man-system 


2 


compatibility are now the team, system, and mission. As can 
be easily seen, the basic premise of MOAT is unchanged in 
that the man-ayasatem compatibility is still being evaluated. 
In short, is the arrangement of equipment in a space "user 
friendly”? Note that even though an arrangement of 
equipment within a work space is being specifically 
addressed,  man-system compatibility subsumes the equipment 
arrangement within the work space. The basic contention is 
that the best operators and best subsystems in a poorly 
designed space may be less effective than operators with 
less ability and a less وہ‎ System that is in a space 
that is well designed for mission accomplishment. 
MOAT uses the term operability to reflect how the man, 
&ystem, and mission interact during ا‎ accomplishment. 
By definition, operability reflects (1) the amount of 
effort required by the operator in task accomplishment, (2) 
the degree of subsystem technical effectiveness in aiding 
the operator in task accomplishment, and (3) how important 
the task is for mission success. (Ref. 2:p. 19) 
This can be redefined slightly to indicate (1) the amount of 
effort required by the operators (team) in task 
accomplishment (task difficulty), (2) the degree that 
equipment arrangement aids the operators (team) in task 
accomplishment (arrangement effectiveness), and (3) how 
important the task is for mission success (task 


criticality). 
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B. TASK ANALYSIS 

The purpose for Task Analysis is to determine those 
tasks and subtasks needed to successfully perform the 
mission of the Module. Without specifying the tasks 
performed within the EW Module, it would be difficult, if 
not impossible, to determine layout effectiveness. 
task/subtask identification forms the basis of both the Link 
Analysis that is discussed later and the MOAT technique of 
Operability Analysis. Each task and subtask that operators 
perform will be examined and fit into the larger picture of 
module mission. The effort within the EW Module can be shown 
to be divided hierarchically: Module mission, operator 
tasks, and operator subtasks. This hierarchy is divided in 
the following manner: the aggregate of the subtasks 
comprises the individual task and the aggregate of the 
individual tasks comprises the mission. 

The mission of the EW Module is to conduct Electronic 
Warfare which includes Electronic Warfare Support Measures 
(ESM) and Electronic Warfare Counter-Measures (ECM). This 
entails attempting to deny any potential enemy the 
exploitation of the electromagnetic spectrum while 
preserving it for our own use. The EW Supervisor (EWS) is 
responsible for providing timely evaluated EW information, 
EW data, and EW control (to the rest of the battle group 
when so designated as EW Control Ship). This is accomplished 


by three operators and three work stations COL R= 1. SEIL, 
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and NTDS). The tasks and subtasks that are performed within 
the workspace are listed in Table 1. While the module 
mission delineates the overall responsibility for the 
module, the operator tasks are the first major subdivision. 


These are the tasks that each operator must accomplish at 


his workstation in order to contribute to mission 
accomplishment. The operator subtask is a further division 
of the operator tasks. These aggregate together for 


ESM/ECM. These are listed in Table 1 and were drawn from 
various sources and confirmed by the EW Module personnel. 
Each workstation and, therefore, each operator has a piece 
of the "puzzle" and only by putting them together can any 
sense be made out of the parts. In this case, as so many 
others, the whole will be greater than the sum of the parts. 

Note in Table 1 that there are actually five positions 
listed: EW Supervisor, WLR-1 operator, SLQ-17 operator, NTDS 
operator, and Status Board keeper. During normal ateaming 
conditions, one of the three position operators is also the 
EW Supervisor and, therefore, he has a dual role to play. 


Additionally, there is no Status Board keeper during normal 


steaming watches. During Condition One, General Quarters, 
the Module is manned with five people. Therefore, the Task 
Analysis considered the more complicated situation of 


General Quartera. 
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TABLE 1. OPERATOR TASKS AND SUBTASKS 


EW Supervisor 


Task: 1.1 Direct ESM search 


1.1.1 Assign search parameters to SLQ-17 

2 Assign search parameters to WLR-1 

3 Assign ESM sensor report responsibilities  -- own ship 
4 Assign ESM sensor report responsibilities -- force 

-> Initiate manual ID request - ship 

6 Initiate manual ID request - force 

7 Monitor automatic correlations/associations, (SLQ-17) 


* 
(» 
(n 
^ 


1.2 Report/Disseminate EW Information 





Subtasks 

1.2.1 Report evaluated EW information 

1.2.2 Provide EW recommendations 

1.2.3 Update status board near NTDS console 
1.2.4 Brief/debrief embarked Airwings 

1.2.5 Navigation by passive EU 


Task: 1.3 Counter Hostile Environment 





a 
.1 Promulgation of ECM employment criteria 


NTDS Operator 


Task: 2.1 Collect and enter EW data into NTDS 
Subtaaka 

Enter manual ID information into NTDS 
Enter manual ESM/NTDS track associations 
Perform triangulation of ESM bearing lines 
Enter EW fixea 

Advise operators of bearing resolution 
Evaluate externally reported ESM bearings 
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Task: 2.2 Report/Disseminate EW Information 
Subtask 

2.2.1 Report evaluated EW information 

2.2.2 Update atatus board near console 


SLQ-17 Operator 


Task: 3.1 Conduct ESM Search 





Subtask 

3.1.1 Monitor automatic correlations/associations 
3.1.2 Establish operating modes of SLQ-17 (ESM) 

3.1.3 Enter detection and response parameters (ESM/ECM) 
3.1.4 Monitor environment on NTDS console 

3.1.5 Evaluate displayed data 


en 


OPERATOR TASKS AND SUBTASKS (continued) 


3.2 Report/Disseminate EW Information 


sk 


Report evaluated EW information 
Provide ESM/ECM data to NTDS 
Monitor entry of EW data into NTDS 
Update status board 


.3 Counter Hostile Environment 


stablish ECM operating Modes 


S 
sk 
Engage targets with ECM 
E 
Assist in promulgation of ECM employment criteria 


TABLE 1. 
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WLR-1 Operator 


Task: 4.1 Conduct ESM Search 
Subtasks 
4.1.1 Search assigned bands 





4.1.2 Analyze intercepted signals 

94.1.3 Check intercepts for images/harmonics 
4.1.4 Accurately DF intercepted signals 
4.1.5 Assist in evaluating ECM 

Task: 4.2 Report/Disseminate EW Information 
Subtasks 

4.2.1 Provide ESM data to NTDS 

4.2.2 Report evaluated EW information 

4.2.3 Update status board near position 
4.2.4 Log all intercepts 

Task: 4.3 EMCON 

Subtasks 

4.3.1 Monitor EMCON 

4.3.2 Report violations of EMCON 

4.3.3 Log violations of EMCON 

4.3.4 Monitor MUTE 


EW Status Board 


1 Maintain Status Boards 


dvise operators of any information received 


Á 

aks 

Communicate with operators 
A 

Update status boards 
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C. LINK ANALYSIS 

Link Analysis is a technique that will provide the 
information needed to produce an acceptable arrangement of 
men and machines in a system (Ref. 3:p. 204]. The idea is 
that the “best arrangement” can be found only by optimizing 
different types of links that are important in the 
particular system being designed. By way of definition, a 
link is a connection between (a) an operator and a machine 
or (b) two operators (Ref. 3:p. 204]. These links may be 
visual ‘(such as an instrument scan), functional or manual 
Chand to control» or verbal (communications). 
Inefficiencies are present when links are comparatively 
long, crossing one another, blocked, or outside optimal 
visual or reach envelopes. The links are produced from the 
task analysis and illustrate all the  operator-required 
functional, visual, and communication tasks. Link Analysis 
can be applied to all scenarios involved during all 
operational and emergency conditions (Ref. Sip. 205 amd 
Ref.4]. Link Analyses are normally of two types: panel 
layout and tactical compartment or multiple operator work 
area. With the development and procurement of individual 
subsystems (i.e., WLR-1, SLQ-17, etc.) a certain amount of 
panel layout link analysis has been done. However, little 
if any has been done on the combination of systems arranged 


in a workspace (in this example, the EW Module). Hence, the 
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Link Analysis will necessarily be of the latter type (i.e., 
multiple operator work area). 

The multiple operator work area type of link analysis is 
dependent on the correlation matrix. Beginning with the 
correlation matrix and an area layout, all interactions 
(links?) required to perform a particular functional task are 
examined in terms of the frequency with which they occur and 
their criticality. If the criticality is assigned k 
numerical value, it may be multiplied by the frequency in 
order to obtain a weighted link value. The work area is 
overlaid with the weighted links permitting a picture of all 
the interactions taking place within the system being 
analyzed. The system design can then be modified to shorten 
the distance between the workstations that are connected by 
the weighted linka [Ref. 51. 

Figure 3 contains the correlation matrix for the EW 
Module in CARL VINSON. A correlation matrix is a figure 
that provides an indication of the links between two 
operators, positions, or between an operator and a position. 
Usually a criticality associated with the particular links 
is included in the matrix. In Figure 3, only the links of 
interest are listed. The figure is read by selecting the 
two entries for which links are desired and reading 
diagonally down from the top one and diagonally up from the 
bottom one until the intersecting diamond is reached. The 


diamond contains both the particular links between the two 
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entries and the criticality of those links. The notation 
along the right side of the figure indicates what links are 
between each component of the module. A separate notation 
provides an indication of the criticality of that particular 
link: most critical, critical, and least critical. The most 
critical link is one that is absolutely essential to 
accomplish the mission. A critical link is one that 
prevents severe degradation in the accomplishment of the 
mission and the least critical link has a small impact. It 
remains only to multiply the links by a frequency of 
operation to obtain a Weighted Link Value. The Weighted 
Link Value will be obtained from the results of the Link 
Analysis Questionnaire. 

The type of links are communication, visual, and manual. 
The communication links are further subdivided into internal 
and external. The internal communication links are the 
voice interaction between operators while the external 
communication links are those voice and/or electronic links 
with other modules, persons, and/or platforms. The visual 
links can alao be divided into internal and external. The 


internal are concerned solely with those links between the 


Operator and his equipment or console. The external are 
those between the operator and other consoles and/or 
equipment. Similarly, the manual links, internal and 


external, are between the operator and his equipment or 


console and the operator and other equipment and/or 


consoles, respectively. Since we have already considered 
that link analysis may have been done on individual systems 
and our concern is for the multiple operator work area, we 
Ph not be concerned with the internal ۰ and internal 
manual links. This leaves just four links analyses to be 
done; the internal and external communications, the external 
visual, and external manual links. 

When the internal communication links are considered, it 
is noted that there are links between the EW Supervisor and 
both the WLR-1 and the SLQ-17 operators to promulgate 
orders. Next, موی‎ are links back to the EU Supervisor 
when one or the other have found either the signal of 
interest or something else that may be of interest. There 


are also links between the WLR-1 operator and the SLG@-17 


operator. This last may be queries for information about 
their particular equipment set up or the passing of 
information to directing the other’s search. The links 


between all three may be in the form of equipment status or 
failure. The internal communication links are shown in 
Figure 4. 

The external communications involve primarily the EW 
Supervisor with any of the following: TAO, INTEL (the 
Intelligence center), SURFACE, AIR, Trackers, ASW cCAnti- 
Submarine Warfare) Module, SSES (Ships Signal Exploitation 
Space), FLAG, FWC (Force Weapons Coordinator), or SWC 


(Ship’a Weapons Coordinator). 
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The external visual links involve all of the operators 
viewing the status boards and each other’s workstation. 
These external visual links are important for the additional 
information they provide to the operator. The visual 
reference to the other workstation builds an internal 
working model of the environment within the operator’s mind 
thus enabling him to more quickly fit new data into the 
tactical picture and anticipate subsequent events. Without 
this interaction, effective Electronic Warfare control can 
not be attained. 

Figures S and 6 show the external visual links. Figure 5 
shows visual links between operators and Figure 6 indicates 
visual links between the operators and the status boards 
(there are only two status boards that can be clearly seen). 

The final link that will be considered is the external 
manual link. Although there is no requirement for an 
operator to control more than one workstation, there are 
Some external manual links that must be addressed. For 
example, all those equipments that are part of the EW Module 
for which there is no manning authorized will fall under 
this category. The AN/SSQ-82 MUTE is a prime example. MUTE 
is a monitor device that requires no manning and only 
cursory glances to ensure that it is functioning correctly. 
When adjustment is needed due to Force or ship EMCON 
changes, an external manual link for one of the operators 


takes place. Another example is the SLQ-17 UYK-20 computer. 
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This equipment is not adjacent to the SLQ-17 and one of the 
operatora (normally the SLQ-17 operator) may need to reload, 
reboot, or reconfigure the system in the event of a casualty 
or normal operations. Figure 7 shows the external manual 


linka. 


D. OPERABILITY ANALYSIS 

In the introduction to this section, it was stated that 
کے ےہ تحت‎ Analysis was comprised of MAU and scaling 
theory. MAU is a Bayesian-oriented decision-making 
paradigm. There are three major aspects of the MAU model 
which are particularly important to this application. First, 
the basic structure principle in MAU is hierarchical 
decomposition. The mission is broken down into hierarchical 
grouping of tasks and subtasks. The model provides the 
atructure and rules necessary to investigate and integrate 
the interrelationships of all these tasks and  subtasxks. 
second, the definition of utility used in the MAU model 
allows for the optimum evaluation of alternatives which is 
dependent upon the selection of a single criterion. This 
means that multidimensional outcomes must be transformed 
into a single figure of merit such as utility, system worth, 
system effectiveness, Or, as in this application, 
operability. Third, a acaling of the selected criterion. 
The scaling methodology used in this application, as in 


MOAT, was conjoint measurement. 
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Recall that operability can be viewed as a function of 
task criticality, operator workload, and space 
effectiveness. Therefore, when considering each task from a 
operability standpoint, each task that is performed has some 
combination of these three dimensions. There is a 
difficulty in assessing the degree of each attribute and 
combining them into a meaningful measure of operability. 
Since this can not be assessed directly by objective 
methods, the scaling methodology of conjoint measurement was 
devised to assess space operability subjectively. The 
problem of scaling tasks in dimensions of criticality, 
frequency, and system effectiveness has been successfully 
solved by using objectively anchored rating scales 
(Ref. 2ip. 20]. Therefore, a similar rating scale procedure 
seemed suitable in this instance. 

The major difficulties involved with this approach are 
those of measuring the degree of operator effort (or watch 
section effort) and the layout effectiveness. This was to 
be expected, however, since not only were these different 
from any known previous study but also involved 
interactions on a higher scale than that experienced before. 
There is a substantial correlation in rating of task 
difficulty and subsystem effectiveness. The attempt to 
solve the rating scale problem is accomplished by dividing 
it into two separate ratings. On the F/A-18 program it was 


desirable to have two ratings; one with respect to pilot 
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workload CPW) and one with respect to the technical 
effectiveness (TE) of the subsystem (Ref. 2:p. 20]. A 
Similar approach is attempted here with operator workload 
(OW) cit is assumed that this can be directly translated 
into watch section workload) and space effectiveness (SE). 
This application of Operability Analysis is concerned with 


assessing the task criticality, the operator workload, and 


the space effectiveness of a module layout. All are values 
on an ordinal scale. Two of these, space effectiveness and 
operator workload, need to be upgraded, through some 


methodology, to an interval scale in order to aggregate them 
over all tasks to achieve an overall measure of Module 
operability. To this end, conjoint measurement and its 
associated scaling procedures seemed suitable for a 
transformation to the desired unidimensional interval level 
scale. It is here that the delta method was employed. What 
conjoint measurement and the delta method do is allow 
separate rating of OW and SE (despite their mutual 
dependency) to be obtained and be combined in such a manner 
that a one-dimensional scale, having interval properties, is 
created. This scale might just as well be called the 
combined  OW-SE scale. Rating on this scale can then be 
plugged into the | MAU model. An assessment of the 
effectiveness of the EW Module layout with respect to a 


certain subtask can be determined. Aggregated together, 
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these will provide an indieatiaen af the overall 


effectiveness of the layout upon mission accomplishment. 


E. QUESTIONNAIRES 

Link Analysis, Task Analysis, and Operability Analysis 
will be completed by a series of two questionnaires. The 
first questionnaire (the Link Analysis Questionnaire) was 
targetted at Link Analysis and provided the frequency 
component for a completed Link Analysis. The second 
questionnaire (the Operator Subtask Questionnaire) confirmed 
the Task Analysis that went into building it and also 
provided the raw data needed to perform the  Operability 
Analysis. From the Operability Analysis came the assessment 
of the Total Module Operability (TMO). 

The questionnaires, as shown in Appendix A, were 
designed to do two things. First, the Link Analysis 
Questionnaire gave a general idea of the type and degree of 
deficiencies in the space in terms of link deficiencies. 
Second, ۴ focused the operators’ attention on the 
arrangement of the work space and any deficiencies that were 
there. It was hoped that the questions brought into sharp 
relief the difficulties for which the operator unconsciously 
compensates during the mission. By highlighting these 
deficiencies through the Link Analysis series of questions, 
the detailing of them in the Operator Subtask 


Questionnaires, hopefully, provided good human engineering 
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data with which to evaluate the Module Operability of the EW 
Module. 

The Link Analysis Questionnaire was designed to determine 
the frequency of the various links in the EW Module. 
Combined with the correlation matrix that indicates 
criticality, this questionnaire determined the weights of 
the various links. This weighting indicated the most 
heavily used links. This, in turn, can focus attention on 
deficiencies in these links. A possible example of this 
might well be the abnormally long internal communication 
link between the NTDS console and the WLR-1 position. 

The Operator Subtask Questionnaire contained a section 
requiring an assessment for each of the forty-nine subtasks 
delineated in Table 1. The subtasks were drawn from a 
variety of sources (including USS CARL VINSON Combat 
Direction Center doctrine) and verified by the EW operators 
prior to the administration of the questionnaire. This 
assessment was the culmination of the Task Analysis and the 
beginning of the Operability Analysis. In the Subtask 
Assessment, each subtask on the questionnaire was evaluated 
by the EW operators with regards to Operator Workload, Space 
Effectiveness, and Criticality of the Subtask to the overall 
mission accomplishment. 

The Operator Subtask Questionnaire also contained the 
Ranking Matrix, where combinations of the various degrees of 


Operator Workload (OW) and Space Effectiveness (SE) were 
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TABLE 2. RANKING MATRIX 
In the following matrix, the blocks are ranked from 
best to worst (1 to 16). The lowest numbered block is the 
intersection of the best of the rows and columns. The number 
two block is next best, and so on. Note the arrows and the 
phrases associated with them. Design means the design of 
the layout or arrangement. 
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ranked. This was of the utmost importance since the 
conjoint measurement of the subtask assessment and Module 
operability depended upon it. Table 2 contains a blank 
Ranking Matrix. The EW personnel were asked to rank the 
intersections from best to worst for the “typical” subtask. 
It was assumed that the rank order for the matrix would vary 
little from subtask to subtask. Helms found this to be true 
(Ref. 2:p. 34]. This may have been the most difficult part 
of the questionnaire and the EU operators were forced to 
draw upon all their knowledge and previous experience in 
order to produce a rank order that was meaningful and 
replicable. This matrix, the intersection of two ordinal 
scales (QU and SE), is part of conjoint measurement. The 
Ranking Matrix was expanded and an interval scale 
constructed via a linear expansion know as the delta method. 
This resulted in an interval scaling from O to 100 and was 
used to evaluate the total Module operability. The delta 
method of converting two of these ordinal scales to an 
interval scale is described in Appendix C. 

Using this interval scale, the intersection of any 
particular set of Operator Workload and Space Effectiveness 
values on the returned Operator Subtask Questionnaire gives 
a predetermined Operability Value between O and 100. An 
Operator Workload value between one and four inclusive 
served to identify a column while a Space Effectiveness 


value identified a row. The intersection of the row with 
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the column indicated the assessment of that particular 
subtask by an operator. For every subtask, this Operability 
Value was obtained for each rater and the mean and standard 
deviation were calculated. This mean value represented the 
Operability Value for that subtask. 

| The remaining ordinal scale is that of the Criticality. 
There was no attempt to convert this to an interval scale. 
Although operators’ skills might vary, causing significant 
deviations in the ratings from rater to rater, there should 
be only one standard for the criticality of a subtask as it 
relates to mission accomplishment. This single measure of 


criticality was taken to be the mean of the criticality 


ratings. The Operability Value was multiplied by the 
criticality resulting in a Weighted Operability Value. A 
Weighted Deficit Value was computed as (100 - Operability 


Valued) multiplied by the Criticality of the subtask. 
Whereas the Weighted Operability Value will give an 
indication of the “goodness” of the layout for a particular 
subtask, the Weighted Deficit Value gives an indication of 
how much improvement is required to optimize Module 
Operability for a particular subtask. 

The Link Analysis Questionnaire was given approximately 
one week before the Operator Subtask Questionnaire. It was 
hoped that the brief exposure to the first questionnaire 


increased the accuracy of the second. 


S0 


IV. RESULTS 


To test this application, a suitable platform was 
required. The Electronic Warfare Module on an U.S. Navy 
aircraft carrier was selected. The particular ship, USS 
CARL VINSON  «CVN-70), was chosen for three reasons: 
availability, accessibility, and familiarity.  CARL VINSON 
had just returned from a seven month cruise and was in a 
stand-down period and, so, available. The ship’s homeport, 
Alameda, Ca., was readily accessible for the test. Finally, 
the ship’s layout was familiar enough to the test director 
to allow a minimum amount of time to be spent on the ship 
and, therefore, lessen the impact upon the ship’s daily work 
and schedule. 

There were limitations to the scope of testing. First, 
the test was not done at sea which produced two limitations. 
In regards to Link Analysis, operator usage of the various 
links and the associated frequencies could not be monitored. 
This was considered to be a major limitation in regards to 
only the Link Analysis portion of the test. The 
compensation for this was the Link Analysis Questionnaire 
concerning the frequency of link usage. A minor limitation 
concerned the inability to observe the actual Subtasks and 


ascertain the criticalities under actual conditions. This 
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was compensated by the Operator Subtask Questionnaire, which 
was considered adequate. 

A further limitation was the small number of valid 
responses for the tica LN There were three valid 
responses for the Link Analysis Questionnaire, five for the 
Rank Ordering portion of the Operator Subtask Questionnaire, 
and from five to seven for the Subtask Assessment portion of 
the Operator Subtask Questionnaire. While these numbers are 
small from a statistical point of view, they can not be 
discounted. The limited sample size should be an inducement 
for further testing. Furthermore, the sample size for any 
aircraft carrier will never be much greater than about 
twelve due to manning levels. The sample size was seven due 


to leave and various schools but included the personnel with 


the most experience. In may be argued that not testing 
other platforms is a limitation. However, since no two EW 
Modules on U.S. aircraft carriers are alike, the lack of 


multiple testing is a moot question. 

The test was conducted in the EW Module of USS CARL 
VINSON (CVN-70O). The Module was used so that the personnel 
could refresh their memory with regards to the layout as 
they evaluated the subtasks in relation to the layout. 

The guidance given to the EW personnel before and during 
the test stressed that they could ask any question they 


wished of anyone they wished. They were encouraged to 
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confer with each other about the workload, effectiveness, 


and criticality. 


A. LINK ANALYSIS 

The results of the Link Analysis were taken from the 
Link Analysis figures and from the Link Analysis 
Questionnaire. The questionnaire was produced from the Link 
Analysis figures and the Task Analysis in order to determine 
the frequency that these links were used. The EW operators 
on USS CARL VINSON were asked to estimate how many times 
during a standard eight (8) hour watch they utilized the 
links. The Link Analysis Questionnaire is listed in 
Appendix A and the results of the Link Analysis is shown in 
Table 3. 

1. Link Analysis Figures 

The most critical links were assessed to be the 

communication links between operators and the visual links 
between positions. The criticality of the links were chosen 


to reflect mission accomplishment and the frequency of usage 


confirmed the criticality. There were four links considered 
in the Link Analysis: internal communication, external 
communication, external visual, and external manual. Of 


these four, the two most important links are the internal 
communications and external visual. This is because the 
external communication will generally involve only one 


operator (the EW Supervisor/NTDS operator) and there is 
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TABLE 3. LINK ANALYSIS BY POSITION 


EC - External Communications; 


EV - External Visuai; EM - External Manuai 


od 


Weighted 
Position and Tasks Frequency Link — Criticality — Link Vaiue 
WLR-1 Operator: 
i. Talk/comaunicate with the SLG-17 operator? 16.667 IC 3 58.08 
2. View the presentation on the SLQ-17 console? 71. 333 EV 3 289 
3. Talk/communicate with the NTDS operator? 44, 020 IC 3 132. 23 
4, View the presentation on the NIDS console? 73: ڈگک‎ EV 3 ES 
5. View the NTDS Status Board (SB)? 33. 333 EV i duds 
6, View the WLR-1 Status Board? 33, $53 EV d 66.67 
7. Update the WLR-1 Status Board? 1. 667 EM e 3, 35 
8. Check (visually) the SLQ-17 computer? 31, 009 EV 1 31. 09 
3. Reboot, reset, or work with the SLG-17 computer? 1. 657 EM 1 1087 
13. Check MUTE? 3. 002 EV e 6. 20 
li. Change any settings on MUTE? 9. 657 EM 2 1, 34 
SLO-17 Operator: 
i, Talk/communicate with the NTDS operator? 25. 000 It 3 162, 2€ 
2, View the presentation on ihe NTDS console? 3.3 EV 3 209ئ2‎ 
3. Talk/communicate with the WLR-1 operator? 71.667 iC 3 2 9 
$, View the oresentation on the wLR-1 console? 48. 333 EV 3 144, 33 
5. View ¿ne WL.R-1 Status Board? 23. 228 EV 3 59. 22 
6. View the NTDS Status Board? SS EV 3 gs. 39 
7. dodate t^e NTDS Status Board? 3. 200 EM 1 9.2 
8. Check (visualiy) the SLO-17 computer? 26. 667 EV e 92. 34 
3. Reboot, reset, or work witn the SLü-i7 computer? 4. 0923 EM 3 2000 
13. Check MUTE? 1. 029 EV 1 1,00 
:i. Change any settings on “UTE? 9. 567 EM Í à. 67 
NTDS Üperator/EW Supervisor: 
is salx/combunicate with ine SLU-17 operator? 45. 667 [C 3 lab. dd 
d. View the presentation on the SiG-17 console? 65.667 EV 3 197. 29 
3j. lal&/communicate with :*5e WLR-1 operator? fence IC 3 dibus 
6. View i-e Dresentation on the WLR-i console? 68. 667 EV ٤ :82 0 
d View 352 mr-i Status Board? 45. 008 EV 3 135. 22 
6. View t^e NTDS Status Boarc? 48, 333 EV 3 144. 99 
7, Jodate tne NTOS Status Board? 16.600 EM 3 48, 22 
8. inecx (visuaily) the SL@-17 computer? 34, 667 EV i 34. 67 
3. Reboot, reset, or work with tne SLü-17 computer? 4, 229 EM ! 4.30 
id. Check MUTE? 6. 667 EV 1 6.67 
ii. Change any settings on MUTE? 4, 280 EM { 4,22 
12. Communicate outside the Module? 200304 EC 3 iGo. Jo 


MEY: If - Internal Communications: 


little requirement for manual links outside of ones own 
position. An external communications link example is the 
link between the EW Supervisor/NTDS operator and the 


communications that enable him to communicate outside the 


Module. However, this requires that the operator rise from 
his seat to communicate. As a remedy, the NTDS operator 
uses a hand mike that hangs down near his console. This is 


a partial solution because he still needs to rise from his 
seat to. select another station on the communication box. 
Additionally, the hand mike hanging so close to his console 
presenta a clutter problem. 

Note that the communication and visual link between 
the NTDS operator (EW Supervisor) and the WLR-1 operator is 
the longest and partially blocked. The links between the 
WLR-1 operator and the NTDS operator and SLQ-17 operator are 
long, allowing him to view very little of the environment. 
The WLR-1 operator’s visual links are very long and the 
parallax effect severely degrades his observation. Note the 
long link lines between the SLQ-17 and NTDS positions and 
the WLR-1 Status Board, and the WLR-1 operator and the NTDS 
Status Board. Finally, note the very long external visual 
links to the SSQ-82 MUTE and that they cross. MUTE is 
required to be checked periodically for faults or changes in 
the various monitor boxes. The distance is great enough 


between MUTE and the rest of the Module that only the WLR-1 
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operator can effectively monitor it. However, this requires 
considerable movement on the part of the WLR-1 operator. 
2. Link Analysis Table 

The frequency of the various links were determined by 
the Link Analysis Questionnaire. The ideal way to determine 
link frequency is to count the actions/link interactions 
during the watch. Since this was not possible, the 
questionnaire approach was chosen. The Link Analysis is 
intended here to focus attention at the links that are used 
most often. The frequency of link usage is multiplied by 
the weight (criticality) of the link and an indication of 
its relative importance is determined. 

When the links associated with the WLR-1 operator 
are considered, it can be noted the longest links are the 
internal communication and external visual links between him 
and the SLQ-17 and NTDS positions. These links are also the 
most critical and the most frequently used. The average 
number of times the operator tries to view the NTDS console 
is 7299923297 Yet this console is the furthest away (see 
Figures 4 and 5). The WLR-1 operator communicates more with 
the NTDS operator for two reasons. Many times the NTDS 
Operator is also the EW Supervisor. The fullest picture of 
the entire environment of surface, subsurface, and air 
contacts is present on the NTDS. The other large frequency 


usage is the visual links for the presentation on the SLQ-17 
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console. This console is only slightly closer than the NTDS 
OM 

In the case of the SLQ-17 operator, the first six 
entries in Table 3 are the ones with the greatest 
criticality and the highest frequency of use. The high 
criticality and frequency associated with checking the SLQ- 
17 computer is understandable since the SLQ-17 operator is 
specifically trained to know what to look for on the 
computer face. Note that the SLQ-17 operator views the 
presentation at the NTDS console much more than that at the 
WLR-1 position. It can be seen from Figure 5 that these 
anal visual links between the SLQ-17 and the NTDS are 
much shorter than between SLQ-17 and the WLR-1. At the same 
time, the SLQ-17 operator communicates more with the WLR-1 
operator than with the NTDS operator. This suggests that 
the SLQ-17 operator gets a better picture of the environment 
from the NTDS but better information concerning the 
environment from the WLR-1. 

The NTDS operator/EW Supervisor are combined because 
many times the EW Supervisor will man the NTDS console for a 
major portion of the watch. This is necessary because all 
the external communications are at or near the NTDS console. 
Note the large frequency and high criticality associated 
with communications outside the Module Cexternal 
communications link). There appears to be a reversal of 


interaction between the NTDS operator/EW Supervisor and the 
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WLR-1 and SLQ-17 positions. He views the  SLQ-17 console 
moe than communicates with the operator but talks more to 
the WLR-1 operator than views the WLR-1 displays. Recall 
from Figures 4 and S that both the internal communications 
and the external visual links between NTDS and WLR-1 are 
very long. Additionally, note how much he looks at the WLR- 
1 Status Board even though it is the furthest away (Figure 
6). 

The Link Analysis is important since it serves to 
indicate which links are long, important, and possibly 
overworked. As such it can be used as a starting point in 
the redesign of a layout by showing which links need to be 
reduced in length. The Link Analysis results should also 


support the results of the Operability Analysis. 


B. OPERABILITY ANALYSIS 

The Operator Subtask Questionnaire was divided into two 
parts: the Subtask Assessment and the Ranking Matrix. The 
Subtask Assessment was given first. The criteria for this 
evaluation and the test itself are given in Appendix A. 

The second half of the Operator Subtask Questionnaire 
was the Ranking Matrix. All returned valid rankings (n=3) 


were entered into the matrix and a mean determined for each 


block and the matrix numbered accordingly. The standard 
deviation was calculated in case of a tie. This matrix with 
the mean rank values and the standard deviation is 
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illustrated in Table 4. The resultant rank matrix is shown 
in Table 3. 

Next this rank ordering was converted to an interval 
scale. This was done by reversing the order of the 
numbering so that the best of the Operator Workload and 
Space Effectiveness is #16 and the worst is #1 (see Table 
cam Using this asa base, the delta method of linear 
expansion was used to determine an interval scale. See 
Appendix C for a brief description, example of the delta 
method, and the final work sheet for this application. 

Table 7 shows the result of the delta method which is the 
desired interval scale. The results of the delta method 
were normalized by dividing all the blocks by the highest 
value in the block; in this application it was 102. Table 8 
is the normalized interval scale for this application. 

The Operability Value was weighted (multiplied) by the 
mean assessed Criticality of that particular Subtask to 
derive the Weighted Operability Value. The Weighted 
Operability Value has the potential to range from an 
absolute minimum of O (0x1) to an absolute maximum of SOO 
(100x520. The range noted was 14.14 to 418.87. 

The Weighted Deficit Value gives an indication of how 
much improvement is needed to optimize Layout Effectiveness 
for a particular Subtask. The greatest Weighted Deficit 


Value was 485.00 while the least was 28.09. The Weighted 
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NORMALIZED INTERVAL SCALE 


TABLE 8. 


Deficit Value can range from 500 to 0. The larger the 
number, the greater the amount of improvement is needed. 

The Total Module Operability for this particular EW 
Module was computed to be 39.2 x. This computation is as 
follows. There were 49 Subtask evaluated. The summation of 
the criticalities of these Subtasks in order to accomplish 
the mission was 200.84. By assuming a perfect layout, we 
can multiply by 100 to obtain a maximum score of 20,084. 
Next the Weighted Operability Values were summed to obtain 
the actual score of the Module of 7872.31. When the actual 
score is divided by the maximum, an indication of the 
effectiveness of the layout is obtained. 

Table 9 contains an ordering of the Subtasks by 
cumulative weight. This was determined by dividing the 
Weighted Deficit Value by the optimum layout effectiveness 
to determine how much the deficit each Subtasks comprises. 
These were then ranked from most to least. This table gives 
an indication of which Subtasks should be improved first in 
order to achieve the most cost effective approach to 
improving the Module. 

Table 9 contains the rank ordering by cumulative 
weights, the Subtask number, a brief description of the 
Subtask and its associated position, the operators polled 
with their evaluation of the Subtask in terms of Operator 
Workload and Space Effectiveness converted to an interval 


scale, the operability value (the mean of the operators’ 
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TABLE 9. RANK ORDER OF SUBTASKS BY CUMULATIVE WEIGHT 


Operability Deficit Cum lative 
No. Subtask Description (Pos) 01,02 3 Q4 05 06 G7 Vaive Vaive Criticality Weignt Totals 
1 1.2.1 Rowe EW Info (Sup) 21 4 9 9 9 9 à 2,98 97.28 865.088 i37 2,97 
2 1.22 Prov EW Recon (Sup) a 9 89 9 86 8 0 019» HAC EMT i137 7.94 
3 41.1 یہ5‎ Agsqn Bnds (WLR) 4 9 8 $921 9 à 1 37.999 &— 35.008 4.4! 1] 
4 422 Rørt EJ Info (WLA) 3 $2383 سے‎ 8 21 8 7.87 92.143 3.908 21.77 1358 
3 42.1 Pry EDN - NTDS (WLR) 2 4 4 0 9 9 à 100 37.988 5.714 1,74 19.42 
6 41.4 OF Signals (WLR) 21 9 4 B Y Y 11.296 88.714 8 26 23 
7 41.2 Analyze Sigs (WLR) 4 8 $8 $01 9$ 8 14 286 85.714 5,088 i31 25.55 
8 31.1 Communicate (SB) 21 24 9 21 8 8ج‎ O1 1&5. 83.4527 4 571 ix 1 
9 21.1 Enter ID?s (NTDS) S 2 9 94. 9 8 م8‎ 97 9.143 0 22 32 
19 4.2.4 Log Intercepts (WLR) 4 89 9 978026 9 1349 86.571 4571 2124 36,47 
11 21.4 Enter EN Fix (NTDS) A» 9 9211080 9 0 22143 77.837 3,008 3219 39.66 
12 21.2 Enter ES Info (NIDOS) 45 @ @ 2k 8 9 8 33.877 98.143 — 4,716 1238 $2.74 
13 S.1.3 Update SB (SB) دہ‎ 9 9 4 210121 28083 79.711 4.714 is 532 
14 2.1.5 Beng Resolution (NDS) 58 21 @ 45 21 9 21 22714 76.288 4.429 217 M33 
i$ 2.1.3 Tragle ESN Beng (NTDS) 21 8 383١ 2138 8 کہ‎ 28 735.388 29 272 Si 
16 1.1.8 Man ID Request (SP! 2 A س بر لي‎ 21 — 22,088 78.008 8 2% 33 
17 111.4 Assign ESN Seen (Sup) B A» 9 21 B 0 4 24429 73.271 4.308 28 B55 
18 3.1.2 Advise Mocule (SB) A» 21 21 901 8 84 Th 24,429 73.371 4. 300 248 8.83 
19 221 Report EN Info (NTDS) 43 61 @ SB BD 21 4 34,714 65.286 54.371 24 61.27 
cB 1.25 Nav by ESM (Sup) 21 214.21 90217, 17.508 2.008 21.308 2.37 63,64 
21 3.24 Update SB (17) e1 Aj 21 at at af at 7 7183 21371 2,298 63,24 
2 1.1.3 Assign ESM reso (Sup  — 45 A 43 2 — 7$ 4,888 5.388 4.308 2,06 — 57.99 
23 1.1.5 Mam ID Rqust--hip (Sup) S8 A 21 4$ -- i6 --  i4.008 49.298 528 2,06 X 69.95 
26 3.3.5 Evaluate Data (17) MS 48 45 21108 21 4S 56.008 54,388 4.571 2% X 71.98 
eS 11.23 Enter Parameters (17) 64 34 S4 3d 21 34 49 45,286 54.714 54,429 1.998 — 73.96 
e$ 4.3.4 Monitor MITE (WLR) A HAHAH SB 21 H.4e9 1.714 1.98 TR 
27 1.2.4 De/Brisf Airminq (Sup) 34 ?21 3 01 ة8‎ 21 74 29,286 78.714 2286 13 77.32 
298 43.1 Monitor CON (WLA) S8 21 61 B o1 61 A3 46,429 ae oF) 4.286 1.88 79.79 
29 4.3.2 Rore DON (MLR) 8ك‎ 4 6 33 21 6 6 MIR S.457 1 557 1.76 81.46 
3 1.31 EDI Criteria (Sup) 58 s 38 21 o 20 47.538 4.308 1.73 ب‎ 7 
31 1.1.2 Assign Searen (Sup) BS HH 21 70 73 49.714 2.26 4.143 1:11 8A, 2 
2 i1.1  Momitor futoCrritn (0170 61 73 54 73 3 73 2 2,388 143.308 4,286 1.69 36.61 
3 21.6 Eval ESM Beng (NTDS) 78 74 74 38 21 43 SB 7.143 42.857 4.286 L — 88.11 
Y 222 Update SB (NTDS) 43 74 75 38 21 43 AS 31.716 48.286 23.571 1.44 8.2 
3S 433 Log Violations (LR) 3 SB 34 46 7€ 3 5ھ‎ 49,286 3714 0 1.23 77 
5ذ‎ 1.23 Update SB (Sup) oS 58 “8 S3 33 61 658,143 39.837 23,714 Let — 31239 
37 12.1 Report EN (17) 86 188 26 86 21 86 Y 65,429 33.571 4.۵86 1.18 31.18 
3 3.22 Provide Data (17) d6 86 74 7h SB 7B 24 67.429 22, 371 4. 388 1.37 AZ 
39 3.3.2 Estab EDN Modes (17) 18 198 108 8 45108 @ 71.714 28.286 — 4.3571 1,086 29 
48 4.1.5 Assist Eval ELM (WR) 3 3 3 108 3 46 SB 18.571 439ء51‎ 2286 2.6% 63 
41 3.1.2 Estab Op Moaes (17) 8 108 198 198 38 735 43 8-1 19.714 23,837 @61 97.35 
42 1.1.1 Assign SL0-17 (Sup) 78 198 198 108 B 58 73 98,143 19.8537 2,167 ac 3173 
43 3.3.3 Assist ELM Employ (17) S3 198 199 108 SB 108 ¿8 82,286 17.714 133 048 6 
44 3.23 Momitor GW Entry (17) 738108 36 74 46 108 74 78.571 21.49 2,571 245 6231 
45 3.3.1 Engage Trgts-<DM (17) 74 198 86 74 198 108 38 88.857 11.143 4.714 8 تھ‎ 3874 
i6 3.1.4 Monitor Errvrrart (17) 96 198 190 106 36 108 45 88,143 11.857 4,988 «39 39:11 
47 4.2.3 Update SB (WLR) 58 108 108 36 C8 108 7^4 30.286 17.714 2,429 aS 08 
48 1.1.7 Monitor SLO-17 (Sup) 386 109 198 198 79 36 — IZ 9.667 3.667 13 7 
49 4.1.3 Check Images (WLR) 78 198 198 106 S8 198 56 87.714 12,286 2,286 1123 100.28 
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evaluations), the deficit value (100 - operability value), 
the mean criticality, the cumulative weight or percentage of 
the total deficit that that particular Subtask comprises, 


and the total percentage. 
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V. DISCUSSION AND RECOMMENDATIONS 


There has been no attempt to ascertain what Weighted 
Deficit Value or Weighted Operability Value is acceptable. 
This isa beyond the scope of this effort. The purpose has 
been to identify which areas are in need of improvement and 
what areas should be addressed first in order to realize the 
greatest amount of improvement for a given effort. To 
answer the question of what Weighted Deficit or Operability 
Value is acceptable will call for additional research 
targeting the Subtasks individually to a greater detail than 


was attempted here. 


A. LINK ANALYSIS DISCUSSION 

The Link Analysis results show that there is only one 
position that might be considered acceptable in relation to 
the lengths of ita links. This is the SLQ-17 position. 
This can be seen in part from the relatively good showing 
that the SLQ-17 console position had in comparison to the 
other two positions. The SLQ-17 operator can easily view 
what is displayed on the NTDS console and, without excessive 
movement, MES the WLR-1 displays. He is within good 
viewing distance of the NTDS Status Board and his own SLQ-17 
computer. The viewing distance to the ULR-1 position and 
its associated Status Board are rather long, but still 


viewable. Because of its relatively good positioning in 
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relation to the rest of the Module, SLQ-17 entries were much 
lower in Table 9. This would indicate that the layout 
actually promotes increased operator compensation since the 
other positions did not score as LL A score of 39.2% is 
an indication of a poor the Module layout contributing کت‎ an 
increased operator compensation burden. Were the module 
layout better, the operators would have felt much better 


about the Module and the score would have been higher. 


B. OPERABILITY ANALYSIS DISCUSSION 

Several observations can be made from Table 39. First, 
the SLQ-17 appears to be the best position in the EW Module 
since its first entry is in twenty-first place in the table 
and most of the entries are at the bottom of the table. 
Almost 27x of the possible improvements can be made in the 
first seven entries and these are just for the EW Supervisor 
and the WLR-1 operator. Note that the criticalities of 
these Subtasks are the highest. In other words, these 
Subtasks which are very critical are poorly supported by the 
layout, relative to the less critical Subtasks. Most of the 
lower criticalities are associated with Subtasks that have a 
relatively good layout effectiveness. 


It can be reasonably argued that Module Operability of 


39.2% is not sufficient for an EW Module. What can not be 
argued is how much improvement is enough. Nor can it be 
extrapolated from this study what improvement a 


67 


rearrangement can result in. However, it can be seen that 
improvement can be made in certain areas, as is indicated by 


careful perusal of Table 3. 


C. EXTRAPOLATION 


Further, this approach can be used for possible 
extrapolations. For example, comparing Figure 1 and Figure 
2, Similarities are noted. They have the same arrangement 


of positions (i.e., from left to right,  WLR-1, SLQ-17, and 
NTDS). The positions are arranged in “straight line" type 
of layout. This resulted in a low Layout Effectiveness 
rating for USS CARL VINSON. It may be readily conjectured 
that another arrangement would work better, namely, a 
“crescent” shaped layout with the NTDS in between WLR-1 and 
SLQ-1 and the supervisor's position raised and directly in 


back of the NTDS operator. 


D. CONCLUSION 

It can be concluded that the present configuration of 
the EW Module on USS CARL VINSON does not result in an 
optimal utilization of this Module in terms of EW mission 
accomplishment. Further, there is a real need to assess the 
layout operability of the warfare modules onboard U.S. Naval 
combatants. This thesis has provided one way in which a 


measure of the effectiveness of a particular layout can be 
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determined. Although this was a limited test, indications 
are that this approach works, and further testing is 
warranted. 

Building a new layout is urged with the hopes that it 
may prove by testing to be better than the last one, using 
the Link Analysis and Operability Analysis illustrated in 
this work. What is significant and useful from the Link 
Analysis is that = improvement in layout design should 
probably start with ensuring that the critical links are not 
overly long or taxed beyond their limit. Any improvement to 
the layout design should take into account these critical 
links to reduce them to their optimum and any changes must 
not adversely affect the links since in that case any gain 
in layout design may be cancelled by a loss in link 
utilization. By conducting tests at landbased test sites, 
the risks of error are reduced. By the utilization of mock- 
ups and fleet inputs, the risks can be reduced even further. 
The result is a more effective layout enhancing mission 


accomplishment. 


E. RECOMMENDATION 

It is recommended that a land based test facility be 
established that would incorporate the ideas, 
recommendations, and methods indicated in this thesis as a 


first step to upgrading our combat workspaces. 
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APPENDIX A 


OPERATOR SUBTASK QUESTIONNAIRE 
GENERAL INSTRUCTIONS 

The purpose of UNUM questionnaire is a subjective 
evaluation of the layout effectiveness of the EW Module for 
use in an algorithm to determine, in an objective sense, the 
effectiveness of the layout in accomplishing the mission of 
the Module. To do this there is a series of  subtasks 
differentiated by operator that must be assessed in terms of 
operator workload per subtask, space effectiveness per 
subtask, and criticality of the subtask toward overall 
mission accomplishment. What is required is to make this 
assessment based on your experience and expertise. There is 
no time limit, you may ask questions of anyone you wish, and 
you should go and look at the Module to make sure of your 
answers especially if you are unsure of some of the 
questions concerning movements. There are no right or wrong 
answers, but try to be as precise as you can. A scenario, 
hopefully similar to your recent operations in the Sea of 
Japan, has been constructed. For each of the subtaska on 
the next page, mark with an "Xx", the description that best 
describes the Operator workload COW) and the space 
effectiveness (SE). If the arrangement of the space has 


little or no effect on subtask accomplishment, then it would 
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rate the highest (4). Conversely, if the layout or 
arrangement of the space negatively impacts subtask 
accomplishment, then it would rate a (1). Give your 
assessment of the criticality of the subtask in relation to 
the overall mission accomplishment. The descriptions of 
criticality, operator workload, and space effectiveness are 


listed on a separate sheet. 


SCENARIO 

This scenario begins with the assumption of the watch 
by a particular section 

They are the on-coming watch section in the EW module 
of a NIMITZ class aircraft carrier that is steaming in the 
open ocean with six escorts. The escorts are one VIRGINIA 
class cruiser, two SPRUANCE class destroyers, one OLIVER 
HAZARD PERRY class frigate, one LOS ANGELES class submarine, 
and an oiler. There are heightened tensions world-wide with 
a probable confrontation between the two super-powers. 
There is a Soviet task group within 200 NM. The task group 
is comprised of a KIEV class aircraft carrier, a KIROV class 
cruiser, a SOVERMENYY class destroyer, two KRIVAK III class 
frigates, a SLAVA class destroyer, and three auxiliaries. 
Additionally, ECHO ILI, VICTOR ILI, and OSCAR class 
submarines are known to be in the area but unlocated for the 


past twelve hours. A Mod-KASHIN is the tattletale for the 


1 


Battle Group. Both forces are within range of Soviet air 
power. 

General Quarters is not set, but a heightened Condition 
III steaming watch is manned. There has been a momentary 
lapse of 400 Hz power and the NTDS is being reloaded. The 
SLQ-17 needs to be reloaded and reprogrammed. As the NTDS 
is brought on the line, the WLR-1 operator is told to 
recheck the past entries in his log and verify that they are 
still active. After 15 minutes, the WLR-1 operator reports 
that he has intercepted several new signals. One is an 
airborne mapping and reconnaissance radar. One appeared to 
be a brief intercept of a submarine radar. Another is an 
air search radar and the last is a missile acquistion radar. 

The NTDS air trackers report jamming on both long range 
and 3-D air search radars. 

The SLQ-17 alarms and displays hostile missile symbols 
from both the suspected direction of the SOviet task group 
and two angles 3O degrees either side of the task group. 

Deck Launched Interceptors are airborne within one 
minute. 

The EW operator at the NTDS console is entering  ESM 
bearing lines and attempting to identify unknown contacts. 
The SLQ-17 operator shifts operation of the ECM portion to 
automatic as the TAO frees weapons. The WLR-1 operator is 
attempting to search the known hostile missile homing radar 


ranges to facilitate identification. General Quarters is 


Z2 


sounded. The TAO orders EMCON to be set for battle and the 
WLR-1 ےت‎ selects EMCON D on MUTE. A quick check of 
both the NTDS scope and that of the SLQ-17 indicates that 
the number and direction of the inbound unknowns do not 
match. The EW watch section tries to match the emerging 
identification from the WLR-1 and SLQ-17 to both the SLQ-17 
and the NTDS presentations. 

If you have trouble envisioning this scenario, recall 
the Sea of Japan operations on your last deployment and 


consider the signal environment and tactics you saw then. 


73 


OPERATOR WORKLOAD, SPACE EFFECTIVENESS, and CRITICALITY 


WORKLOAD/COMPENSATION/INTERFERENCE (Mental and Physical) 


(1) (2) (3) (4) 
Workload Extreme Workload High Workload Moderate Workload Low 
Compensation Extreme Compensation High Compensation Moderate Compensation Low 
Interference Extreme Interference High Interference Moderate Interference Low 


SPACE EFFECTIVENESS 


(1) (2) (3) (4) 
Inadequate Performance Adequate Performance Layout Enhances Spe- Layout Desicn 
Due to Layout Achievable; Layout cific Task Accomplish- Integrates 

Sufficient to Spec- ment Multiple Tasks 
ific Task 


CRITICALITY: How important is it that the operator/team be 
able to perform this task as compared to the other 
tasks in successfully completing the mission? 


Scale Value: 


(1) Of very small importance. Ability to perform this 
task as compared to other tasks in this duty is unimportant, 


or almost unimportant, in order to successfully complete the 
mission of the Module. 


(2) Qf small importance. This task within this duty is 
lessa important than most tasks required to successfully 
complete the mission of the Module. 


(3) Of moderate importance. This task within this duty is 
about as important as most tasks required to successfully 
complete the mission of the Module. 


(4) Of substantial importance. This task within this duty 
is more important than most tasks required to successfully 
complete the mission of the Module. 


(50 Of extreme importance. This task within this duty is 


extremely important in order to successfully complete the 
mission of the Module. 


7d 


(4) 


(4) 


(4) 


(4) 


-- Own ship 


(4) 


(4) 


-- force 


(4) 


(4) 


EW Supervisor 


Task: 1.1 Direct ESM search 
Subtasks 
1.1.1 Assign search parameters to SLQ-17 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) (3) 


Criticality: 
1.1.2 Assign search parameters to WLR-1 


Operator Workload: 


(1) (2) C39 


Space Effectiveness: 


(1) (2) (3) 


Criticality: 


1.1.3 Assign ESM sensor report responsibilities 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) C3) 


Criticality: 


1.1.4 Assign ESM sensor report responsibilities 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) C39 


Criticality: 
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(SLQ-17) 


(4) 


(4) 


(4) 


(4) 


(1) (2) (3) 


1.1.5 Initiate manual ID request - ship 


Operator Workload: 


Space Effectiveness: 


(1) (2) (3) 


Criticality: 
1.1.6 Initiate manual ID request - force 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) (3) 


Criticality: 


1.1.7 Monitor automatic correlations/associations, 


(4) 


(4) 


(4) 


(4) 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) (3) 


Criticality: 


i 


ask: 1.2 Report/Disseminate EW Information 
Subt 
DT 





asks 
.2.1 Report evaluated EW information 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) (3) 


Criticality. 
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(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(3) 


(3) 


1.2.2 Provide EW recommendations 


Operator Workload: 


(1) (2) 


Space Effectiveness: 


(1) (2) 


Criticality: 


1.2.3 Update status board near NTDS console 


(3) 


(3) 


(3) 


(3) 


(3) 


(3) 


Operator Workload: 


(1) (2) 


Space Effectiveness: 


(1) (2) 


Criticality: 
1.2.4 Brief/debrief embarked Airwings 


Operator Workload: 


(1) (2) 


Space Effectiveness: 


(1) (2) 


Criticality: 
1.2.5 Navigation by passive EW 


Operator Workload: 


(1) (2) 


Space Effectiveness: 


(1) (2) 


Criticwrility: 


Task: 1.3 Counter Hostile Environment 
Subtasks 
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1.3.1 Promulgate ECM employment criteria 


Operator Workload:  . J-J.— _. 09س بآ سسسسسسہ مہ‎ - 
(1) (2) (3) (4) 


Space Effectiveness: aes DEO 
(1) (2) (3) (4) 


Criticality: 


NTDS Operator 


Task: 2.1 Collect and enter EW data into NTDS 
Subtasks 
2.1.1 Enter manual ID information into NTDS 


Operator Workload: NEUEN — 2 
(1) (2) a (4) 


Space Effectiveness: .نے سے‎ A 
(1) C2) (3) (4) 


7۷۶ 1> >1 7ب 
Enter manual ESM/NTDS track associations‏ 2.1.2 


Operator Workload: —€—€—————— — ٦ 
(1) (2) (3) (4) 


Space Effectiveness: —— ——————  L— 
(1) (2) (3) (4) 


Criticality:i 
2.1.3 Perform triangulation of ESM bearing lines 


Operator Workload: کے کے‎ ٢ 
(1) (2) (3) (4) 


0 ى .تےے- نے Space Effectiveness:‏ 
)4( )3( )2( )1( 


Criticalitv: 
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(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


2.1.4 Enter EW fixes 


Operator Workload: 





C1) (2) (3) 
Space Effectiveness: . .———»^— 
(1) (2) (3) 
Criticality: 
2.1.9 Advise operators of bearing resolution 
Operator Workload: NL O 
(1) (2) (3) 
Space Effectiveness: lll  — 
(1) (2) (3) 
Criticality: 
2.1.6 Evaluate externally reported ESM bearings 
Operator workload: جادہ -.>ے‎  :7 70 9 
(1) (2) 632 
Space Effectiveness: ہے لات ود جک‎ 
(1) (2) (3) 
Creiticality : 
Task: 2.2 Report/Disseminate EW Information 
Subtask 
2.2.1 Report evaluated EW information 
Operator Workload: A AAA 
(1) (2) (3) 
Space Effectiveness: E S 
CI (2) (3) 
Criticality: 
2.2.2 Update status board near console 
Operator Workload: NENNEN. 
(1) (2) (3) 
Space Effectiveness: 27ے کے‎ 5۳ 
(1) (2) (3) 


Criticality: 
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SLQ-17 Operator 


Conduct ESM Search‏ ک1 


4 





ask: 3 
ubtask 
.1.1 Monitor automatic correlations/associations 


Y JU 


Operator Workload: |. |. | . 
C1) (2) (3) (4) 


Space Effectiveness:  _س_. سس‎ 
(1) (2) (3) (4) 


Criticality: 
3.1.2 Establish operating modes of SLQ-17 (ESM) 


Operator Workload: E +009 -- 00ء‎ 
(1) (2) (3) (4) 


ص س۹ت 1تت Space Effectiveness:‏ 
)4( )3( 2( )1( 


Criticality. 
3.1.3 Enter detection and response parameters (ESM/ECM) 


Operator Workload: ET TS a eee 
(1) (2) (3) (4) 


Space Effectiveness: x ——— 
(1) (2) (3) (4) 


Criticality: 
3.1.4 Monitor environment on NTDS console 


Operator Workload: oce 
(1) (2) (3) (4) 


Space Effectiveness: ILL EAN A EA 
CI (2) (3) (4) 


Critreality: 
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(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


3.1.5 Evaluate displayed data 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) (3) 


Eriticality: 


-] 


.2 Report/Disseminate EW Information 





ask: 3 
ubtask 
.2.1 Report evaluated EW information 


U) |U 


Operator Workload: 


CIO (2) C3) 


Space Effectiveness: 


(1) (2) C3) 


Criticality: 
3.2.2 Provide ESM/ECM data to NTDS 


Operator Workload: 


GT.) (2) (3) 


Space Effectiveness: 


er) (2) (3) 


Criticality: 
3.2.3 Monitor entry of EW data into NTDS 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) (3) 


Criticality: 
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(4) 


(4) 


(d) 


(4) 


(4) 


(4) 


criteria 


(4) 


(4) 


3.2.4 Update status board 


Operator Workload: ____ . —4. —. ____ 
(1) (2) (3) 


Space Effectiveness: ^. —  -— 
(1) (2) (3) 


Criticality: 


Task: 3.3 Counter Hostile Environment 
Subtask 
3.3.1 Engage targets with ECM 


Operator Workload: E AMA 
(1) (2) (3) 


Space Effectivenesa:  — — —  — — 
(1) (2) (3) 


Cricricality: 
3.3.2 Establish ECM operating Modes 


مہہ مہ ہہ _ Operator Workload:‏ 
)3( )2( )1( 


Space Effectiveness: _...۰ ےہ_۔‎  —^"— 
(1) (2) (3) 


Criticality: 
3.3.3 Assist in promulgation of ECM employment 


Operator Workload: A TE > 
(1) (2) (3) 


Space Effectiveness: کے‎ س١س‎ 9, 7 
(1) (2) (3) 


Criticality: 
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(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(3) 


(3) 


(3) 


(3) 


(2) 


(2) 


(2) 


(2) 


WLR-1 Operator 


Task: 4.1 Conduct ESM Search 
Subtasks 


4.1.1 Search assigned bands 


Operator Workload: 
(1) 


Space Effectiveness: 
(1) 


Criticality: 


4.1.2 Analyze intercepted signals 


Operator Workload: 
(1) 


Space Effectiveness: 
(1) 


Criticality: 


4.1.3 Check intercepts for images/harmonics 


(3) 


(3) 


(3) 


(3) 


(2) 


(2) 


Operator Workload: 
(1) 


Space Effectiveness: 
(1) 


Seriticality: 


4.1.4 Accurately DF intercepted signals 


(2) 


(2) 


Operator Workload: 
(1) 


Space Effectiveness: 
(13 


Criticality: 
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(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(3) 


(3) 


4.1.5 Assist in evaluating ECM 


Operator Workload: 
(1) (2) 


Space Effectiveness: 
(1) (2) 


Criticality: 


Task: 4.2 Report/Disseminate EW Information 


(3) 


C3) 


(3) 


CS» 


(3) 


C3) 


Subtasks 
4.2.1 Provide ESM data to NTDS 


Operator Workload: __۔‎ _____ 
C1) (2) 


Space Effectiveness:  JAJ  . ^—  -. 
(1) (2) 


Criticality: 
4.2.2 Report evaluated EW information 


Operator Workload: | Gk. = | .-. 
ep (2) 


Space Effectiveness: 
(1) (2) 


Criticality: 
4.2.3 Update status board near position 


Operator Workload: 
(1) (2) 


Space Effectiveness: 
(1) (2) 


Criticalivey. 
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(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(4) 


(3) 


(3) 


(3) 


(3) 


C39 


(3) 


(3) 


63) 


(3) 


(3) 


(2) 


(2) 


(2) 


(2) 


(2) 


(2) 


(2) 


(2) 


(2) 


(2) 


4.2.4 Log all intercepts 


Operator Workload: 
(1) 


Space Effectiveness: 
(1) 


criticality: 
Task: 4.3 EMCON 


Subtasks 
4.3.1 Monitor EMCON 





Operator Workload: 
(1) 


Space Effectiveness: 
(1) 


Gritteality: 
4.3.2 Report violations of EMCON 


Operator Workload: 
(1) 


Space Effectiveness: 
(1) 


Criticality: 
4.3.3 Log violations of EMCON 


Operator Workload: 
(1) 


Space Effectiveness: 
(1) 


Criticality: 
4.3.4 Monitor MUTE 


Operator Workload: 
(13 


Space Effectiveness: 
(1) 


eritıcality: 
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(4) 


(4) 


EW Status Board 

Task: 5.1 Maintain Status Boards 
Subtasks 

5.1.1 Communicate with operators 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) (3) 


Criticality: 


5.1.2 Advise operators of any information received 


(4) 


(4) 


(4) 


(4) 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) CZ) (3) 


Criticality; 
9.1.3 Update status boards 


Operator Workload: 


(1) (2) (3) 


Space Effectiveness: 


(1) (2) (3) 


Criticality: 
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Rating Matrix Cell Rank Order 
For the following matrix, rank the blocks from best to 
worst (1 to 16). The lowest numbered block is the 
intersection of the best of the rows and columns. The number 
two block is next best, and so on. Note the arrows and the 
phrases associated with them. Design means the design of the 
layout or arrangement. Do not think of the layout of one 
workstation, such as the WLR-1 or SLQ-17, but of the entire 
EW Module. Think of the scenario already presented in order 
to properly consider the workload. Ask any questions you 
want or talk among yourselves or go and look at the layout. 
Multiple Tasks 
Integrated 
Layout Enhances 
Specific Task 
Aecowolishment 
Adequate 
Performance 
Achievadie 


(Layout Sufficient) 


Inadequate 
Performarice 2.8€ to 
Layout 0300۴06 جن‎ 
pensate For Su- 
Par Performance 


Effectiveness Improving 


acrkloac At Workloaa Can- Workload Siigntly worxlcad As 
Critical Level;  sicerabiy Higher Higher Than 8۸: 1٥٤0٤ 
0ئ٣۲٣‎ T3an SntiCipated; Anticioates; No interference; 
Very Excessive X C:nrenzililh Yoderate No Lompersatior 
hich; c.tevference  Cempensation; 
major “ieor interference 





WORKLOAD IMPROVING 
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LINK ANALYSIS QUESTIONNAIRE 


WLR-1 Operator: 


How many times in an eight hour watch do you: 


Talk/communicate with the SLQ-17 operator? 
View the presentation on the SLQ-17 console? 
Talk/communicate with the NTDS operator? 
View the presentation on. the NTDS console? 
View the NTDS Status Board (SB)? 

View the WLR-1 Status Board? 

Update the WLR-1 Status Board? 

Check (visually?) the SLQ-17 computer? 


Reboot, reset, or work with the SLQ-17 computer? 


10. Check MUTE? 


Change any settings on MUTE? 
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11. 


SLQ-17 Operator: 


How many times in an eight hour watch do you: 


the NTDS operator? 


on the NTDS console? 


the WLR-1 operator? 


on the WLR-1 console? 


Board? 


Talk/communicate with 


View the presentation 


Talk/communicate with 


View the presentation 


View the WLR-1 Status 


View the NTDS Status Board? 


Update the NTDS Status Board? 


Check (visually) the SLQ-17 computer? 


Reboot, reset, or work with the SLQ-17 computer? 


10. Check MUTE? 


Change any settings on MUTE? 
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11. 


NTDS Operator/EW Supervisor: 


How many times in an eight hour watch do you: 


the SLQ-17 operator? 


on the SLQ-17 console? 


the WLR-1 operator? 


on the WLR-1 console? 


Board? 


Talk/communicate with 


View the presentation 


Talk/communicate with 


View the presentation 


View the WLR-1 Status 


View the NTDS Status Board? 


Update the NTDS Status Board? 


Check (visually) the SLQ-17 computer? 


Reboot, reset, or work with the SLQ-17 computer? 


10. Check MUTE? 


Change any settings on MUTE? 


11. 


12. Communicate outside the Module? 
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APPENDIX B 


QUESTIONNAIRE RESULTS 
Rating Matrix Cell Rank Order 
In the following matrix, the blocks rank from best to 

worst (1 to 16). The lowest numbered block is the 
intersection of the best of the rows and columns. The number 
two block is next best, and so on. Note the arrows and the 
phrases associated with them. Design means the design of the 
layout or arrangement. 


Multiple Tasks 
Integrated 9 6 E 1 


Layout Enhances 


Specific Task 12 7 4 3 
Accomp] i shment 
Adequate 
Performance 
Rchievabie 14 11 18 5 
(Layout Sufficient) 
Inadequate 
Performance cue to 
Layout, Cannot Com- 16 I5 i3 3 
pensate For Sub- 
Par Performance 
aorxicac At Workload Son- worklcad Siigntly werxlcad As 
Critical Level;  sicerabiy ‘icher Higner Than Ariticicated; 
Compensation Tran Anticipated; anbicigated; NO interference: 
Very Excessive Compensation tocerata No concensat13n 
High; interference  Compersaticn; 
“a jor “incr Interferer.ce 
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RESULTANT INTERVAL SCALE for OQPERABILITY 





122 


5 


72 


Worklcad Rs 
Anticicatec; 
No Interference; 
NO 6٣۴ 


74 


Workload Slightly 
Higher Than 
Anticipated; 
Mocerate 
٠۲٥10958101) 
Minor Interference 
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61 


43 


2] 


Workload Corr 
siceraoly Hicher 
Taan Anticipated; 
Compensation 

High; Interference 
Ma jor 


32 


24 


workload At 
Critical Level; 
Compensation 
Very Excessive 


Multiple Tasks 
Integrated 


Layout Enhances 
Specific Task 
Accomplishment 


Adequate 
Performance 
Achievabie 

(Layout Sufficient) 


Inadequate 
Performance due to 
Layout, Cannot Con- 
pensate For Sub- 
Par Performance 


RESULTS OF ALL OPERATORS 
NOTES: The following results are the Operability Value of 
all responses for each aubtask. Generally, n equalled 7 for 
the subtasks, although there were some with only six 
responses. The Standard Deviation is that of the sample and 
not the population (i.e., the standard deviation was 
calculated using n vice n-1). The Interval Scale shown was 
based on the Ranking Scale which the EW operators provided 
through the questionnaire. The Ranking Can ordinal) Scale 
was then converted to an interval scale by means of the 
Delta Method (a linear expansion). The Criticality listed 
is a mean for the subtask. The Weighted Operability Value 
is simply the Operability Value weighted (multiplied) by the 


Criticality. 


EW Supervisor 


= 


ask: 1.1 Direct ESM search 





Subtasks 

1.1.1 Assign search parameters to SLQ-17 
Operability Value: 80.143 
Criticality: 3167 
Weighted Operability Value: 2533.81 
Weighted Deficit Value: 62.99 


1.1.2 Assign search parameters to WLR-1 


Operability Value: 49.714 
Sriticalaty: 4.143 
Weighted Operability Value: 203.97 
Weighted Deficit Value: 208.33 
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1.1.3 Assign ESM sensor report responsibilities  -- own ship 


Operability Value: 44.000 
Criticality: 4.500 
Weighted Operability Value: 138.00 
Weighted Deficit Value: 232.00 
1.1.4 Assign ESM sensor report responsibilities -- force 
Operability Value: 24.429 
Criticality: 4.000 
Weighted Operability Value: gue 
Weighted Deficit Value: 302.28 
1.1.5 Initiate manual ID request ship 
Operability Value: 40.800 
Criticality: 4.250 
Weighted Operability Value: 173.40 
Weighted Deficit Value: 251.60 
1.1.6 Initiate manual ID request force 
Operability Value: 22.000 
Criticality. 4.000 
Weighted Operability Value: 88.00 
Weighted Deficit Value: 312.00 


1.1.7 Monitor automatic correlations/associations, (SLQ-17) 


90.333 
3.667 
331.25 
35.45 


Operability Value: 
Criticalit». 

Weighted Operability Value: 
Weighted Deficit Value: 


Task: 1.2 Report/Disseminate EW Information 


Subtaska 


1.2.1 Report evaluated EW information 


3.000 
9.000 
15.00 
485.00 


3.000 
9.000 
15.000 
485.00 


Operability Value: 
Criticelity: 

Weighted Operability Value: 
Weighted Deficit Value: 


1.2.2 Provide EW recommendations 
Operability Value: 
Criticalicy: 


Weighted Operability Value: 
Weighted Deficit Value: 
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1.2.3 Update status board near NTDS console 


Operability Value: 60.143 
Criticality: 3.714 

Weighted Operability Value: 223657 
Weighted Deficit Value: 148.03 


1.2.4 Brief/debrief embarked Airwings 


Operability Value: 29.286 
Criticality: 3.286 
Weighted Operability Value: 96.23 
Weighted Deficit Value: 232.37 


1.2.5 Navigation by passive EW 


Operability Value: 17.500 
Criticality: 3.500 
Weighted Operability Value: 6 5 
Weighted Deficit Value: 288.75 


Task: 1.3 Counter Hostile Environment 





Subtasks 

1.3.1 Promulgation of ECM employment criteria 
Operability Value: 32.300 
Criticality: 4.500 
Weighted Operability Value: 236.25 
Weighted Deficit Value: 213.79 


NTDS Operator 


Task: 2.1 Collect and enter EW data into NTDS 
Subtaaka 
2.1.1 Enter manual ID information into NTDS 


Operability Value: 9:857 
Crit icality: 4.500 
Weighted Operability Value: 45.06 
Weighted Deficit Value: 405.64 


2.1.2 Enter manual ESM/NTDS track associations 


Operability Value: 20.286 
Crhticality: 4.714 
Weighted Operability Value: 235.63 
Weighted Deficit Value: 37/3 7 
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2.1.3 Perform triangulation of ESM bearing lines 


Operability Value: 25.9000 
Criticality: 4.429 
Weighted Operability Value: 120.73 
Weighted Deficit Value: 332.18 


2.1.4 Enter EW fixes 


Operability Value: 22.143 
Criticality: 5.000 

Weighted Operability Value: 110.72 
Weighted Deficit Value: 3893.29 


2.1.5 Advise operators of bearing resolution 


Operability Value: 23.714 
Criticality: 4.429 
Weighted Operability Value: 105.03 
Weighted Deficit Value: 337.87 


2.1.6 Evaluate externally reported ESM bearings 


Operability Value: 57.143 
Criticality: 4.286 

Weighted Operability Value: 244.91 
Weighted Deficit Value: 183.69 


Task: 2.2 Report/Diaseminate EW Information 
Subtask 
2.2.1 Report evaluated EW information 


Operability Value: 34.714 
Criticality: 4.571 

Weighted Operability Value: 158.68 
Weighted Deficit Value: 298.42 


2.2.2 Update status board near console 


Operability Value: 51.714 
Criticadreye 3.571 

Weighted Operability Value: 184.67 
Weighted Deficit Value: 172.43 


SLQ-17 Operator 


Taak: 3.1 Conduct ESM Search 
Subtask 
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3.1.1 Monitor automatic correlations/associations 


Operability Value: 292.000 
Criticality: 4.286 

Weighted Operability Value: 222.87 
Weighted Deficit Value: 205.73 


3.1.2 Establish operating modes of SLQ-17 (ESM) 


Operability Value: 80.571 
Criticality: 34837 
Weighted Operability Value: 310.76 
Weighted Deficit Value: 74.94 


3.1.3 Enter detection and response parameters (ESM/ECM) 


Operability Value: 45.286 
Criticality: 4.429 
Weighted Operability Value: 200.57 
Weighted Deficit Value: 242.33 


3.1.4 Monitor environment on NTDS console 


Operability Value: 88.143 
CGriticality: 4.000 
Weighted Operability Value! 352: 97 
Weighted Deficit Value: 47.43 


3.1.5 Evaluate displayed data 


Operability Value: 46.000 
CriticaRity: 4.571 

Weighted Operability Value: 210.27 
Weighted Deficit Value: 246.83 


Task: 3.2 Report/Disseminate EW Information 
Subtask 
3.2.1 Report evaluated EW information 


Operability Value: 66.429 
Gratrcality: 4.286 

Weighted Operability Value: 284.71 
Weighted Deficit Value: 143.89 


3.2.2 Provide ESM/ECM data to NTDS 


Operability Value: 67.429 
criticality: 4.000 

Weighted Operability Value: 269: 72 
Weighted Deficit Value: 130.28 
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3.2.3 Monitor entry of EW data into NTDS 


Operability Value: 71 
Criticality: 2.971 
Weighted Operability Value: 202.00 
Weighted Deficit Value: 35.09 


3.2.4 Update status board 


Operability Value: 24.857 
Criticality: 3.571 
Weighted Operability Value: 88.76 
Weighted Deficit Value: 268.34 


Task: 3.3 Counter Hostile Environment 
Subtask 
3.3.1 Engage targets with ECM 


Operability Value: 88.857 
6 888 99ئ8‎ 1)٣ 4.714 
Weighted Operability Value: 418.87 
Weighted Deficit Value: ڈو ےو‎ 


3.3.2 Establish ECM operating Modes 


Operability Value: 71.714 
Critiestity: 4.571 

Weighted Operability Value: 327.80 
Weighted Deficit Value: 129730 


3.3.3 Assist in promulgation of ECM employment criteria 


Operability Value: 82.286 
Criticality: 3.233 
Weighted Operability Value: 274.26 
Weighted Deficit Value: 39.04 


WLR-1 Operator 


Task: 4.1 Conduct ESM Search 





Subtasks 

4.1.1 Search assigned bands 
Operability Value: 3.000 
Criticality: 5.000 
Weighted Operability Value: 13.00 
Weighted Deficit Value: 485.00 


38 


14.286 
3.000 
71.43 
428.37 


87.714 
2.286 
200.51 
28.09 


4.1.2 Analyze intercepted signals 


Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 


4.1.3 Check intercepts for images/harmonics 


Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 


4.1.4 Accurately DF intercepted signals 


11.286 
3.000 
56.43 
443.57 


48.571 
2.286 

i. O3 
0 7 


3.000 
4.714 
14.14 
457.26 


Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 


4.1.5 Assist in evaluating ECM 


Operability Value: 
Criticafity: 

Weighted Operability Value: 
Weighted Deficit Value: 


Task: 4.2 Report/Disseminate EW Information 


Subtasks 
4.2.1 Provide ESM data to NTDS 


Operability Value: 
Critreelity. 

Weighted Operability Value: 
Weighted Deficit Value: 


4.2.2 Report evaluated EW information 


7 7 
5.000 
39 0 
460.72 


Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 


4.2.3 Update status board near position 


82.286 
2.429 
139.37 
43.03 


Operability Value: 
Griticality: 

Weighted Operability Value: 
Weighted Deficit Value: 
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13.429 
4.571 
61.38 
395.72 


46.429 
4.286 

198.99 
229,61 


44.143 
3.857 

170.26 
215.44 


49.286 
3.000 

147.86 
152.14 


doo 
3.714 

1392.11 
239929 


102573 
4.571 
48.33 
408.77 


4.2.4 Log all intercepts 


Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 


Task: 4.3 EMCON 
Subtasks 
4.3.1 Monitor EMCON 


Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 


4.3.2 Report violations of EMCON 


Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 


4.3.3 Log violationa of EMCON 


Operability Value: 

81121-7۰٤‏ 131 ۶ن 

Weighted Operability Value: 
Weighted Deficit Value: 


4.3.4 Monitor MUTE 
Operability Value: 
Criticality: 


Weighted Operability Value: 
Weighted Deficit Value: 


EW Status Board 


ad 


asks Maintain Status Boards 





Sree 
Subtaaka 
9.1.1 Communicate with operators 

Operability Value: 
Criticality: 

Weighted Operability Value: 
Weighted Deficit Value: 
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5.1.2 Advise operators of any information received 


Operability Value: 24.429 
Criticality: 4.000 
Weighted Operability Value: S772 
Weighted Deficit Value: 302.28 


9.1.3 Update status boards 


Operability Value: 20.289 
Criticality: 4.714 
Weighted Operability Value: 95.64 
Weighted Deficit Value:: 375.76 
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APPENDIX C 


THE DELTA METHOD 

This appendix gives a brief description of the delta 
method, an algorithm for converting ordinal measures on the 
cella of a matrix to a scale with interval properties 
satisfying the conditions of additive conjoint measurement. 
This appendix is essentially the same as that in Ref. 2, 
only being changed to reflect the present application. For 
a further and fuller description, see Coombs (Ref. 6]. The 
method will be briefly described using the matrix in Figure 
CR This matrix is similar in form to the 4x4 matrix used 
in the OW/SE rating matrix developed for the Subtask 


Queationnaire but smaller in aize. 


| | 
| | 
| ۱ 
| | 
| | 
Factor II Q | 3 | 4 
| | 
| | 
| | 
| | 
| ۱ 
| | 


n a سے‎ -— eee ë —  ؛ےصیےہ‎ ei 0 — 
مس‎ m مجح مسو‎ |e m eee 


A B C 
Factor I 


Figure C-1 
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27 Figure  C-1, Factors I and II represent two 
independent measures, and the numbers in the cells of the 
matrix represent an empirical ordering of overall 
performance over combinations of factors I and II. Higher 
numbers represent better overall performance. 

The resulting scales will be interval measures of I and 
II as well as overall performance. Because the scale is 
Mc iva, the measure of overall performance of any cell 
must be the sum of the corresponding row and column scale 
values. Furthermore, the resulting performance measure must 
reflect the ordering of the cells of the matrix. 
Consequently, any set of scale values which provide an 
additive representation for a matrix must simultaneously 
satisfy the equations implied by the additive representation 
and the inequalities specified by the rank ordering of the 
cells of the matrix. Conditions under which a set of linear 
equations and inequalities have a common solution are 
specified mathematically by the Theorem of the Alternative. 
In practice, solutions may be found by using various linear 
programming techniques. The delta method is one such 
technique that is simple enough to be done by hand for small 
matrices. 

The delta method proceeds, in general, as follows. 
Cells in the matrix are initially given arbitrary positive 
scale values (represented by the Greek letter, delta; hence, 


the name)(we will replace delta with the Roman letter d, for 
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ease of computation) which satisfy the equations specified 
by the additive representation. For example, if B were 
assigned the value dj and Q were assigned the value d3, then 
the cell BQ would have the value dM + d3. After initial 
assignments are made, the relationships between scale 02 
and the d’s are changed to take into account the constraints 
given by the rank ordering of the matrix. When the 
procedure is completed and a solution is found, scale values 
are represented by positive linear combinations of the d's, 
such that any choice of positive d’s will lead to scale 
values which satisfy both the equations implied by 
additivity and the inequalities implied by rank order. 

The levels of each factor are assigned values which 
reflect the ordering on that factor. Thus for Factor I, A 
will be assigned a value of O since it is the lowest level 
of the factor. B will be assigned an arbitrary positive 
value of di. Since C is a higher level of performance than 
B it may be expressed as dj * d2 where d2 is an arbitrary 
positive constant representing the difference between the 
Scale values of B and C. Similarly, in Factor II, P, G, and 
R may be assigned values of O, aan and da, respectively. 
Since the overall performance is an additive combination of 
the two factors, the scale value of individual cells may be 


stated in terms of the d’s as has been done in Figure C-2. 
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| | | ۱ 
3+ R | ۹١3+٥4 | d4*d3*d4 I d1+d2+d3tdą4 | 
| | ۱ ۱ 
| | i | 
| | | ۱ 
Factor d3 Q | d3 | d4 *d3 ia tao tas ۱ 
If | | | ۱ 
| ۱ i | 
| | | ۱ 
OP | O ۱ d1 | d4*d2 ۱ 
| ۱ i ۱ 
| | | | 
A B C 
O di di+d2 


Factor I 


Figure C-2 


Alternately, the scale values may be displayed as in 


Figure C-3. this figure represents the work sheet which 
will be used in the algorithm. On the top half of the 
figure are the individual factor scale values. On the 


bottom half are the values of overall performance, listed in 
decreasing rank order. The columns represent the four d's. 
the numbers in the work sheet are the coefficients in the 
equation: 

Scale Value = coef1 x dj * coef2 x d2 * coef3 x d3 * coefa x 
eA. 


A blank indicates a coefficient of zero. 
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p>‏ تا () تن 0 0ه 


E Y AA OY ae ae aD ee se ee ee ee Y A Y Y AY AY A A A ss BF Y ese ss ss as as as 


Q 
0 
2 


wm 
A 
P 


CQ | 1 


AR ۱ 


BQ | 1 
AQ ا‎ 


| 
۱ 
| 
۱ 
| 
۱ 
| 
| 
CP i1 l1 ۱ 
۱ 
۱ 
۱ 
| 
| 
BP | 
۱ 
۱ 


AP 


Figure C-3 


The general procedure involves comparing cells adjacent 


the 


that 


we 


the 


the 


of 


has 


in the rank ordering of the matrix and redefining 
relationship between the scale values and the d's so 
the order of the cells will be preserved for any choice 
positive d's. the cells may be examined in any order; 
this example and the Layout Effectiveness application, 
will start from the lowest performance and proceed to 
highest level. This involves moving from the bottom to 
top of the work sheet. 

The first pair of cells in BP and AP. examination 


Figure C-3 shows that BP is higher than AP. Since BP 
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value dj and AP has value O, it may be concluded that d1 > 
O. This inequality is clearly satisfied for any positive 
value for d1, so it is not necessary to redefine this value. 
It will be recalled that values were assigned to A and B so 
that B would have a higher value than A. 

The second inequality is AQ > BP. Substituting the 
values in this inequality gives d3 5» dj. This inequality is 
not true for all values of d3 and dj. However, since d3 >? 
di, it possible to replace d3 by d1 * d3, for positive dz’. 
Now, for any choice of positive dj and d3'/, the inequality 
d1 + d3* > d21 holds. On the work sheet, d3 is replaced by 
d4 * d3^; that is, in any row with a d3, a dj and a d3’ are 
added and the d3 is deleted. For convenience, and because 
the d3’ column looks exactly like the d3 column, the d3 
column is put where d3 was. This is merely a relabeling of 
columns. Note that as many ‘marks’ as were in the d3 column 
are added to the dj column. The work sheet at this point 
looks like Figure C-4. 

The next inequality, BQ > AQ implies 2d, + d3’% > d1 + 
کت‎ Or di > Q. Again it is not necessary to make any 


changes to satisfy this constant. 
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do nothing 


A | | | | | | | 
B 11 | | | | | | 
E 11 11 | | | | | 
P | | | | | | | 
Q n | P | | i | 
R E | 14 11 ۱ | | 
CR 7+ ET 11 11 | | | 
| | | ۱ | | 
BR 1141 | 7 11 | | | 
| | | | | | | 
cQ 111  !1 11 | | | | 
| | | | ۱ | 
AR 11 | 11 11 | | | 
| | | | | | | 
CP 11 11 | | | | | 
| | | | | ۱ | 
BQ 111 | 11 | | | | 
| | | | | | | 
AQ 11 | 11 | | | | AQ»BP-d3»d| 
| | | | | ۱ | ,d3=d1+d3’ 
BP 11 | | | | | BP>AP=d1>0 
| | | | | 
| | | | 


Figure C-4 


Proceeding up the work sheet, the next inequality is CP 
» BQ. This implies that d2 > d1 * d3. Here, we do the 
same as before, in that we redefine d> as dj + d3% + do’, 
and consequently replacing d2 in every row in which it 
occurs by*t di, daa”, and do". Again d2’ is put in the 
column where d2 was. Consequently, the procedure involves 
relabeling the d2 column d2’ and adding a d1 and a d3’ for 
each d2 in any row in which a d2 appears. The work sheet at 


this point looks like Figure C-S. 
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A | | | | | l ۱ 
B 11 | | | | | | 
C bli ll l1 ۱ | | 
P | | | | | | 
Q 11 11 | | | 
R 11 | 11 11 | | ۱ 
CR LTL 111 11 | | | 
| | ۱ ۱ | ۱ | 
BR LET | 11 11 | | | 
| ۱ | ۱ | ۱ | 
CQ J|] qw T ERI ۱ | | i 
| ۱ ۱ | | ۱ | 
AR r1 i | 1 11 | 1 | 
| ۱ | | | ۱ | 
CP Erl 11 IT | | | | CP>BQ=d>>d1+d3” 
| ۱ ۱ | | ۱ ۱ d22d4*d35'-*d2' 
BQ PIT | |1 | | | | BQ>AQ=d1>0 
۱ | ۱ | | | | do nothing. 
AQ | 1 | Er | | ۱ | AQ>BP=d3>d1 
۱ | | ۱ | | | d3=d1*d3 
BP pr | | | | | I BP>AP=d1>0 
| ۱ ۱ | | ۱ ۱ do nothing. 
AP | | | | | | | 


Figure C-5 


The inequality AR > CP implies a similar inequality 
among d’s and is handled the same way. The work sheet at 
this point is given in Figure C-6. 

The ordering of CQ > AR implies an inequality among tne 
d’s which is handled somewhat differently from the previous 
inequalities. CQ > AR implies that dz + d3’ > da’. Since 
there are two d's on the left side of the inequality, Le 2s 
not possible simply to make the substitution: 

dis tas (= dq” + dj’ .+/d3°". 
Some rows may have different numbers of dz and d3”, so this 


replacement rule would be impossible to implement. 
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The following three step method is used to redefine the 
d’s. First, dą’ is split into two parts, d4q4’’ and ds. 
Since d4 * d3^ » d4^, this division may be arbitrarily done 
so that d1 > d4'* and d3” > dS. The preceding two 


inequalities may be handled by 


| 
AP i 


do 7 


d1 d2^ d3* da” 

A | | ] | | l | 
B 11 | | | | 
C 11 11 11 | | | | 
P | | | | | | | 
Q 11 | 11 | | | | 
R ITI i1 11 1 | | | 
CR I1LI EE 11 | ۱ i 

۱ ۱ ۱ ۱ | ۱ 
BR III | 1 11 l | ۱ 

۱ ۱ | ۱ ۱ | ۱ 
CQ 7 Sp NLL | i | | 

| | ۱ ۱ ! ! ۱ 
AR AS ni 11 | | | AR>CP=dg>d1+d>’ 

| | | | | | ۱ da=d1+*d2'*+da” 
CP 111 1 1 | | i ا‎ CP>BQ=d2>d1*+td3’ 

i | ۱ ۱ | ۱ ۱ a2=d1] +03 cp 
BQ ۲11. ا‎ | | | | BQ>AQ=d1>0 

| | ۱ | | ۱ | do nothings 
AQ 1 | E | | | | AQ>BP=d3>d1 

۱ ۱ | ۱ ۱ i ۱ d3=d1+d3” 
BP 11 | | i | IL BP>AP=d1>0 

| | | | 
| | | i | 


Figure C-6 
previously discussed methods. Thus the three replacements 


are 


LIO 


Now for any choice of positive d’s, CQ > AR is 
satisfied. The work sheet now looks like Figure C-7. 

The methods of handling the remaining inequalities have 
already been discussed. Note that the steps to complete the 
worksheet is (matrix size - 1), or in the example, 8. For 
the Module Operability application there were 15 steps. 

When completed, the top half of the work sheet shows the 
relationship between scale values and the newly defined d's. 


For the example given, the following relationships hold: 


A = 0 
B = Gye + dg” + da’ 
e2. + do’ * 3d3^* + 2d4**” + ds 
P= 0 
Q = d1’ * 2001“ + 80 ھ7٤‎ » ds 
Reece + Q2 *dd3 ^ * 3d4""" + 2d5 
d1 d2. d3 waa ds 
A l ۱ i ۱ | | 
B 11 | | | | | | 
C 11 11 I1 EL 11 | | 
P | | | | | | | 
Q 11 | |1 1 11 | | 
R IT 11 1 Il ٦ ۱ | 
CR ۲۷٢ Li 1L1 111 5 111| 
۱ ۱ | | | ۱ ۱ 
BR UTIL IT 2 TT | | (da4*=da4*' +ds 
۱ ۱ | | ۱ | | )٥1٭*٥‎ ^ed!" 
CQ ھ‎ 77 7 IG ERIT ۱ | CQ»AR-d4*d3'»da" 
| ۱ ۱ | | | ۱ (aa ASOT 
AR ELI I1 1 IIL o ¡| AR>CP=d34>d1+d29” 
۱ ۱ | | | | | da=dirda> Td 
CP ek 11 11 11ا‎ ol | | CP>BQ=d9>d1+d2’ 
۱ | ۱ | ۱ | do-d|*d3'*d2"' 
BQ 111 | 11 HEI I1 | | BQ>AQ=d1>0 
| ۱ | | i ۱ | do nothing. 
AQ I1 | 11 11 ET | | AQ>BP=d3>d1 
۱ | ۱ ۱ | | | dosd]*ds3' 
BP 11 | ۱ |1 | | | BP>AP=d1>0 
| ۱ | ۱ | ۱ ۱ do nothing. 
AP ۱ | | | | | | 


Figure C-7 


ok al 


Any choice of positive d’s will give a scale for Factors I 
and II as well as overall performance which is an additive 
representation, and in which the overall ordering agrees 
with the empirical ordering. Figure C-8 contains the 


resulta of this example. 


d1 d2’ 4-32  da^'"" ds dg d7 dg 
A | | | | ۱ | i | {| = O 
B 11 i 11 11 | | | | | = 3 
C 111 | 21 111 1 | | | | = 9 
p | | | ۱ ۱ | ۱ | 2 0 
Q 11 | 111 |1 por | | ۱ ا‎ - 5 
R 111 E IIA TD 111 | ۱ | | 12 
CR IIIIII IT 7 Md T ] | | | = 20 
BR 1111 11 (5 11111 i | | | | - 15 
CQ 1111 11 IS EER 111 | ۱ | | = 14 
AR 111 11 Pil ۹ 9 08 ٤ 111 | ۱ | | = 12 
CP I111 |1 LT 1ا‎ 11 i | | {| = 9 
BQ Ill | CEIL 111 11 | | I = 8 
AQ 11 | 111 I 1 ۱ | | | s 5 
BP 11 i1 ا‎ | | | | = 3 
AP | | | ۱ | | | i i= Ọ 


Figure C-8 

A common choice of d’s is to make them all equal to 1. 
This choice for d yields a set of scale values which 
represent the set of minimal integers which will produce the 
requires rank order of the matrix cells. 

The completed work sheet for the Module  Operability 
application follows. Note that the numbers within the d’s 
are expressed in arabic numerals for viewing ease and the 


totals are listed on the side. 


112 


O 


22 
35 
47 


24 
40 
59 


102 
90 
87 
77 
25 
71 
62 
29 
99 
47 
46 
40 
393 
24 
Ze 





i | ۱ | | | i | | 
12 13 E: 3 13 11 E: 12 | 
3 5 7 IS E: 12 6 13 i 
i4 I6 | 10 17 16 12 13 | 4 | 
| i ۱ i | | | i | 
12 13 IS | 4 13 I1 14 12 
13 IS 18 l6 IS 12 7 i4 | 
IS 7 111 18 17 3 I3 IS | 
19 113 | 21 15 TS IS 17 9و‎ | 
18 112 118 i13 111 IS 5 18 | 
17 111 118 3 111 | 4 US 18 | 
7 19 15 TI | lO E 13 7 | 
16 110 | 39 11 9 E: 15 7 | 
6 19 [15 11ا‎ I9 13 12 6 ۱ 
IS 18 12 9 18 3 2 6 | 
IS 18 112 9و‎ 17 13 | 10 IS i 
IS 17 IT 18 17 3 19 |S | 
E: 6 | 10 7 I6 12 18 E: i 
E: l6 19 | 7 16 12 13 14 | 
13 IS 13 6 5 12 17 | q | 
3 [5 17 I5 4 12 I6 3 | 
12 13 IS E: 13 I1 2ھ‎ |2 | 
12 13 E: 13 3 ا‎ E: EZ | 

| 


Figure C-9 
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